728x90
반응형
비지도학습 기반 감성 분석
SentiWordNet을 이용한 Sentiment Analysis
- WordNet Synset과 SentiWordNet SentiSynset 클래스의 이해
import nltk
nltk.download('all')
[nltk_data] Downloading collection 'all'
[nltk_data] |
[nltk_data] | Downloading package abc to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\abc.zip.
[nltk_data] | Downloading package alpino to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\alpino.zip.
[nltk_data] | Downloading package biocreative_ppi to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\biocreative_ppi.zip.
[nltk_data] | Downloading package brown to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\brown.zip.
[nltk_data] | Downloading package brown_tei to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\brown_tei.zip.
[nltk_data] | Downloading package cess_cat to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\cess_cat.zip.
[nltk_data] | Downloading package cess_esp to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\cess_esp.zip.
[nltk_data] | Downloading package chat80 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\chat80.zip.
[nltk_data] | Downloading package city_database to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\city_database.zip.
[nltk_data] | Downloading package cmudict to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\cmudict.zip.
[nltk_data] | Downloading package comparative_sentences to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\comparative_sentences.zip.
[nltk_data] | Downloading package comtrans to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package conll2000 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\conll2000.zip.
[nltk_data] | Downloading package conll2002 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\conll2002.zip.
[nltk_data] | Downloading package conll2007 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package crubadan to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\crubadan.zip.
[nltk_data] | Downloading package dependency_treebank to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\dependency_treebank.zip.
[nltk_data] | Downloading package dolch to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\dolch.zip.
[nltk_data] | Downloading package europarl_raw to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\europarl_raw.zip.
[nltk_data] | Downloading package floresta to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\floresta.zip.
[nltk_data] | Downloading package framenet_v15 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\framenet_v15.zip.
[nltk_data] | Downloading package framenet_v17 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\framenet_v17.zip.
[nltk_data] | Downloading package gazetteers to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\gazetteers.zip.
[nltk_data] | Downloading package genesis to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\genesis.zip.
[nltk_data] | Downloading package gutenberg to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\gutenberg.zip.
[nltk_data] | Downloading package ieer to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\ieer.zip.
[nltk_data] | Downloading package inaugural to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\inaugural.zip.
[nltk_data] | Downloading package indian to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\indian.zip.
[nltk_data] | Downloading package jeita to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package kimmo to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\kimmo.zip.
[nltk_data] | Downloading package knbc to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package lin_thesaurus to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\lin_thesaurus.zip.
[nltk_data] | Downloading package mac_morpho to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\mac_morpho.zip.
[nltk_data] | Downloading package machado to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package masc_tagged to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package moses_sample to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping models\moses_sample.zip.
[nltk_data] | Downloading package movie_reviews to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\movie_reviews.zip.
[nltk_data] | Downloading package names to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\names.zip.
[nltk_data] | Downloading package nombank.1.0 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package nps_chat to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\nps_chat.zip.
[nltk_data] | Downloading package omw to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\omw.zip.
[nltk_data] | Downloading package opinion_lexicon to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\opinion_lexicon.zip.
[nltk_data] | Downloading package paradigms to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\paradigms.zip.
[nltk_data] | Downloading package pil to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\pil.zip.
[nltk_data] | Downloading package pl196x to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\pl196x.zip.
[nltk_data] | Downloading package ppattach to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\ppattach.zip.
[nltk_data] | Downloading package problem_reports to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\problem_reports.zip.
[nltk_data] | Downloading package propbank to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package ptb to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\ptb.zip.
[nltk_data] | Downloading package product_reviews_1 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\product_reviews_1.zip.
[nltk_data] | Downloading package product_reviews_2 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\product_reviews_2.zip.
[nltk_data] | Downloading package pros_cons to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\pros_cons.zip.
[nltk_data] | Downloading package qc to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\qc.zip.
[nltk_data] | Downloading package reuters to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package rte to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\rte.zip.
[nltk_data] | Downloading package semcor to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package senseval to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\senseval.zip.
[nltk_data] | Downloading package sentiwordnet to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\sentiwordnet.zip.
[nltk_data] | Downloading package sentence_polarity to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\sentence_polarity.zip.
[nltk_data] | Downloading package shakespeare to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\shakespeare.zip.
[nltk_data] | Downloading package sinica_treebank to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\sinica_treebank.zip.
[nltk_data] | Downloading package smultron to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\smultron.zip.
[nltk_data] | Downloading package state_union to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\state_union.zip.
[nltk_data] | Downloading package stopwords to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\stopwords.zip.
[nltk_data] | Downloading package subjectivity to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\subjectivity.zip.
[nltk_data] | Downloading package swadesh to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\swadesh.zip.
[nltk_data] | Downloading package switchboard to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\switchboard.zip.
[nltk_data] | Downloading package timit to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\timit.zip.
[nltk_data] | Downloading package toolbox to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\toolbox.zip.
[nltk_data] | Downloading package treebank to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\treebank.zip.
[nltk_data] | Downloading package twitter_samples to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\twitter_samples.zip.
[nltk_data] | Downloading package udhr to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\udhr.zip.
[nltk_data] | Downloading package udhr2 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\udhr2.zip.
[nltk_data] | Downloading package unicode_samples to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\unicode_samples.zip.
[nltk_data] | Downloading package universal_treebanks_v20 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package verbnet to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\verbnet.zip.
[nltk_data] | Downloading package verbnet3 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\verbnet3.zip.
[nltk_data] | Downloading package webtext to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\webtext.zip.
[nltk_data] | Downloading package wordnet to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\wordnet.zip.
[nltk_data] | Downloading package wordnet_ic to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\wordnet_ic.zip.
[nltk_data] | Downloading package words to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\words.zip.
[nltk_data] | Downloading package ycoe to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\ycoe.zip.
[nltk_data] | Downloading package rslp to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping stemmers\rslp.zip.
[nltk_data] | Downloading package maxent_treebank_pos_tagger to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping taggers\maxent_treebank_pos_tagger.zip.
[nltk_data] | Downloading package universal_tagset to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping taggers\universal_tagset.zip.
[nltk_data] | Downloading package maxent_ne_chunker to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping chunkers\maxent_ne_chunker.zip.
[nltk_data] | Downloading package punkt to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping tokenizers\punkt.zip.
[nltk_data] | Downloading package book_grammars to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping grammars\book_grammars.zip.
[nltk_data] | Downloading package sample_grammars to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping grammars\sample_grammars.zip.
[nltk_data] | Downloading package spanish_grammars to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping grammars\spanish_grammars.zip.
[nltk_data] | Downloading package basque_grammars to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping grammars\basque_grammars.zip.
[nltk_data] | Downloading package large_grammars to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping grammars\large_grammars.zip.
[nltk_data] | Downloading package tagsets to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping help\tagsets.zip.
[nltk_data] | Downloading package snowball_data to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package bllip_wsj_no_aux to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping models\bllip_wsj_no_aux.zip.
[nltk_data] | Downloading package word2vec_sample to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping models\word2vec_sample.zip.
[nltk_data] | Downloading package panlex_swadesh to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package mte_teip5 to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\mte_teip5.zip.
[nltk_data] | Downloading package averaged_perceptron_tagger to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping taggers\averaged_perceptron_tagger.zip.
[nltk_data] | Downloading package averaged_perceptron_tagger_ru to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping
[nltk_data] | taggers\averaged_perceptron_tagger_ru.zip.
[nltk_data] | Downloading package perluniprops to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping misc\perluniprops.zip.
[nltk_data] | Downloading package nonbreaking_prefixes to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping corpora\nonbreaking_prefixes.zip.
[nltk_data] | Downloading package vader_lexicon to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Downloading package porter_test to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping stemmers\porter_test.zip.
[nltk_data] | Downloading package wmt15_eval to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping models\wmt15_eval.zip.
[nltk_data] | Downloading package mwa_ppdb to
[nltk_data] | C:\Users\pc\AppData\Roaming\nltk_data...
[nltk_data] | Unzipping misc\mwa_ppdb.zip.
[nltk_data] |
[nltk_data] Done downloading collection all
True
from nltk.corpus import wordnet as wn
term = 'present'
# 'present'라는 단어로 wordnet의 synsets 생성.
synsets = wn.synsets(term)
print('synsets() 반환 type :', type(synsets))
print('synsets() 반환 값 갯수:', len(synsets))
print('synsets() 반환 값 :', synsets)
synsets() 반환 type : <class 'list'>
synsets() 반환 값 갯수: 18
synsets() 반환 값 : [Synset('present.n.01'), Synset('present.n.02'), Synset('present.n.03'), Synset('show.v.01'), Synset('present.v.02'), Synset('stage.v.01'), Synset('present.v.04'), Synset('present.v.05'), Synset('award.v.01'), Synset('give.v.08'), Synset('deliver.v.01'), Synset('introduce.v.01'), Synset('portray.v.04'), Synset('confront.v.03'), Synset('present.v.12'), Synset('salute.v.06'), Synset('present.a.01'), Synset('present.a.02')]
for synset in synsets :
print('##### Synset name : ', synset.name(),'#####')
print('POS :',synset.lexname())
print('Definition:',synset.definition())
print('Lemmas:',synset.lemma_names())
##### Synset name : present.n.01 #####
POS : noun.time
Definition: the period of time that is happening now; any continuous stretch of time including the moment of speech
Lemmas: ['present', 'nowadays']
##### Synset name : present.n.02 #####
POS : noun.possession
Definition: something presented as a gift
Lemmas: ['present']
##### Synset name : present.n.03 #####
POS : noun.communication
Definition: a verb tense that expresses actions or states at the time of speaking
Lemmas: ['present', 'present_tense']
##### Synset name : show.v.01 #####
POS : verb.perception
Definition: give an exhibition of to an interested audience
Lemmas: ['show', 'demo', 'exhibit', 'present', 'demonstrate']
##### Synset name : present.v.02 #####
POS : verb.communication
Definition: bring forward and present to the mind
Lemmas: ['present', 'represent', 'lay_out']
##### Synset name : stage.v.01 #####
POS : verb.creation
Definition: perform (a play), especially on a stage
Lemmas: ['stage', 'present', 'represent']
##### Synset name : present.v.04 #####
POS : verb.possession
Definition: hand over formally
Lemmas: ['present', 'submit']
##### Synset name : present.v.05 #####
POS : verb.stative
Definition: introduce
Lemmas: ['present', 'pose']
##### Synset name : award.v.01 #####
POS : verb.possession
Definition: give, especially as an honor or reward
Lemmas: ['award', 'present']
##### Synset name : give.v.08 #####
POS : verb.possession
Definition: give as a present; make a gift of
Lemmas: ['give', 'gift', 'present']
##### Synset name : deliver.v.01 #####
POS : verb.communication
Definition: deliver (a speech, oration, or idea)
Lemmas: ['deliver', 'present']
##### Synset name : introduce.v.01 #####
POS : verb.communication
Definition: cause to come to know personally
Lemmas: ['introduce', 'present', 'acquaint']
##### Synset name : portray.v.04 #####
POS : verb.creation
Definition: represent abstractly, for example in a painting, drawing, or sculpture
Lemmas: ['portray', 'present']
##### Synset name : confront.v.03 #####
POS : verb.communication
Definition: present somebody with something, usually to accuse or criticize
Lemmas: ['confront', 'face', 'present']
##### Synset name : present.v.12 #####
POS : verb.communication
Definition: formally present a debutante, a representative of a country, etc.
Lemmas: ['present']
##### Synset name : salute.v.06 #####
POS : verb.communication
Definition: recognize with a gesture prescribed by a military regulation; assume a prescribed position
Lemmas: ['salute', 'present']
##### Synset name : present.a.01 #####
POS : adj.all
Definition: temporal sense; intermediate between past and future; now existing or happening or in consideration
Lemmas: ['present']
##### Synset name : present.a.02 #####
POS : adj.all
Definition: being or existing in a specified place
Lemmas: ['present']
# synset 객체를 단어별로 생성합니다.
tree = wn.synset('tree.n.01')
lion = wn.synset('lion.n.01')
tiger = wn.synset('tiger.n.02')
cat = wn.synset('cat.n.01')
dog = wn.synset('dog.n.01')
entities = [tree , lion , tiger , cat , dog]
similarities = []
entity_names = [ entity.name().split('.')[0] for entity in entities]
# 단어별 synset 들을 iteration 하면서 다른 단어들의 synset과 유사도를 측정합니다.
for entity in entities:
similarity = [ round(entity.path_similarity(compared_entity), 2) for compared_entity in entities ]
similarities.append(similarity)
# 개별 단어별 synset과 다른 단어의 synset과의 유사도를 DataFrame형태로 저장합니다.
similarity_df = pd.DataFrame(similarities , columns=entity_names,index=entity_names)
similarity_df
tree lion tiger cat dog
tree 1.00 0.07 0.07 0.08 0.12
lion 0.07 1.00 0.33 0.25 0.17
tiger 0.07 0.33 1.00 0.25 0.17
cat 0.08 0.25 0.25 1.00 0.20
dog 0.12 0.17 0.17 0.20 1.00
import nltk
from nltk.corpus import sentiwordnet as swn
senti_synsets = list(swn.senti_synsets('slow'))
print('senti_synsets() 반환 type :', type(senti_synsets))
print('senti_synsets() 반환 값 갯수:', len(senti_synsets))
print('senti_synsets() 반환 값 :', senti_synsets)
senti_synsets() 반환 type : <class 'list'>
senti_synsets() 반환 값 갯수: 11
senti_synsets() 반환 값 : [SentiSynset('decelerate.v.01'), SentiSynset('slow.v.02'), SentiSynset('slow.v.03'), SentiSynset('slow.a.01'), SentiSynset('slow.a.02'), SentiSynset('dense.s.04'), SentiSynset('slow.a.04'), SentiSynset('boring.s.01'), SentiSynset('dull.s.08'), SentiSynset('slowly.r.01'), SentiSynset('behind.r.03')]
import nltk
from nltk.corpus import sentiwordnet as swn
father = swn.senti_synset('father.n.01')
print('father 긍정감성 지수: ', father.pos_score())
print('father 부정감성 지수: ', father.neg_score())
print('father 객관성 지수: ', father.obj_score())
print('\n')
fabulous = swn.senti_synset('fabulous.a.01')
print('fabulous 긍정감성 지수: ',fabulous .pos_score())
print('fabulous 부정감성 지수: ',fabulous .neg_score())
father 긍정감성 지수: 0.0
father 부정감성 지수: 0.0
father 객관성 지수: 1.0
fabulous 긍정감성 지수: 0.875
fabulous 부정감성 지수: 0.125
from nltk.corpus import wordnet as wn
# 간단한 NTLK PennTreebank Tag를 기반으로 WordNet기반의 품사 Tag로 변환
def penn_to_wn(tag):
if tag.startswith('J'):
return wn.ADJ
elif tag.startswith('N'):
return wn.NOUN
elif tag.startswith('R'):
return wn.ADV
elif tag.startswith('V'):
return wn.VERB
return
from nltk.stem import WordNetLemmatizer
from nltk.corpus import sentiwordnet as swn
from nltk import sent_tokenize, word_tokenize, pos_tag
def swn_polarity(text):
# 감성 지수 초기화
sentiment = 0.0
tokens_count = 0
lemmatizer = WordNetLemmatizer()
raw_sentences = sent_tokenize(text)
# 분해된 문장별로 단어 토큰 -> 품사 태깅 후에 SentiSynset 생성 -> 감성 지수 합산
for raw_sentence in raw_sentences:
# NTLK 기반의 품사 태깅 문장 추출
tagged_sentence = pos_tag(word_tokenize(raw_sentence))
for word , tag in tagged_sentence:
# WordNet 기반 품사 태깅과 어근 추출
wn_tag = penn_to_wn(tag)
if wn_tag not in (wn.NOUN , wn.ADJ, wn.ADV):
continue
lemma = lemmatizer.lemmatize(word, pos=wn_tag)
if not lemma:
continue
# 어근을 추출한 단어와 WordNet 기반 품사 태깅을 입력해 Synset 객체를 생성.
synsets = wn.synsets(lemma , pos=wn_tag)
if not synsets:
continue
# sentiwordnet의 감성 단어 분석으로 감성 synset 추출
# 모든 단어에 대해 긍정 감성 지수는 +로 부정 감성 지수는 -로 합산해 감성 지수 계산.
synset = synsets[0]
swn_synset = swn.senti_synset(synset.name())
sentiment += (swn_synset.pos_score() - swn_synset.neg_score())
tokens_count += 1
if not tokens_count:
return 0
# 총 score가 0 이상일 경우 긍정(Positive) 1, 그렇지 않을 경우 부정(Negative) 0 반환
if sentiment >= 0 :
return 1
return 0
review_df['preds'] = review_df['review'].apply( lambda x : swn_polarity(x) )
y_target = review_df['sentiment'].values
preds = review_df['preds'].values
from sklearn.metrics import accuracy_score, confusion_matrix, precision_score
from sklearn.metrics import recall_score, f1_score, roc_auc_score
print(confusion_matrix( y_target, preds))
print("정확도:", accuracy_score(y_target , preds))
print("정밀도:", precision_score(y_target , preds))
print("재현율:", recall_score(y_target, preds))
[[7668 4832]
[3636 8864]]
정확도: 0.66128
정밀도: 0.647196261682243
재현율: 0.70912
VADER lexicon을 이용한 Sentiment Analysis
from nltk.sentiment.vader import SentimentIntensityAnalyzer
senti_analyzer = SentimentIntensityAnalyzer()
senti_scores = senti_analyzer.polarity_scores(review_df['review'][0])
print(senti_scores)
# {'neg': 0.13, 'neu': 0.743, 'pos': 0.127, 'compound': -0.7943}
def vader_polarity(review,threshold=0.1):
analyzer = SentimentIntensityAnalyzer()
scores = analyzer.polarity_scores(review)
# compound 값에 기반하여 threshold 입력값보다 크면 1, 그렇지 않으면 0을 반환
agg_score = scores['compound']
final_sentiment = 1 if agg_score >= threshold else 0
return final_sentiment
# apply lambda 식을 이용하여 레코드별로 vader_polarity( )를 수행하고 결과를 'vader_preds'에 저장
review_df['vader_preds'] = review_df['review'].apply( lambda x : vader_polarity(x, 0.1) )
y_target = review_df['sentiment'].values
vader_preds = review_df['vader_preds'].values
print('#### VADER 예측 성능 평가 ####')
from sklearn.metrics import accuracy_score, confusion_matrix, precision_score
from sklearn.metrics import recall_score, f1_score, roc_auc_score
print(confusion_matrix( y_target, vader_preds))
print("정확도:", accuracy_score(y_target , vader_preds))
print("정밀도:", precision_score(y_target , vader_preds))
print("재현율:", recall_score(y_target, vader_preds))
#### VADER 예측 성능 평가 ####
[[ 6736 5764]
[ 1867 10633]]
정확도: 0.69476
정밀도: 0.6484722815149113
재현율: 0.85064
반응형
'Data_Science > ML_Perfect_Guide' 카테고리의 다른 글
| 8-7. Opinion Review || 문서 군집화 (0) | 2022.01.02 |
|---|---|
| 8-6. 토픽 모델링(Topic Modeling) - 20 뉴스그룹 (0) | 2022.01.02 |
| 8-4. IMDB 영화평 || 지도학습 기반 감성 분석 (0) | 2022.01.02 |
| 8-3. 뉴스그룹 분류 (0) | 2022.01.02 |
| 8-2. Bag of Words (0) | 2022.01.02 |
