Customer sentiment analysis using text mining

Text mining can be used to cull out customer comments from user generated content like website comments to give an idea of how customers feel about the experience they had at a store or with a product. For example www.yelp.com has a lot of comments about customer experiences of a product, service etc. This can be mined say using Oracles text miner to understand keywords which are used to express a sentiment and rank their frequency. Some questions which an organisation can answer using text mining are

1) What are the top 3 keywords which occur frequently online when a sentiment is expressed ?
2) Is their any affinity between choice of keywords & product,age,progression,location etc ?
3) Which are the keywords growing fastest in the last 3 months in terms of frequency and what does that tell us about product or service we deliver and the process of delivering that experience ?
4) Is the overall sentiment trending favorably or is their reason to be concerned ?
5) How are we doing vis a vis competitition from a buzz perspective ?
6) What are the top 3 keywords which are used to express a sentiment about competition online ?
7) How can we create a strategy to respond to what we are hearing based on online buzz / feedback ?

There are few sentiment related key performance indicators which can be used to answer some of the above questions. They are

1) Sentiment velocity : Figuring out the direction of the sentiment
2) Positive Sentiment index : Ratio of positive vs negative sentiments
3) Buzz index : No of entries by source
4) Keyword : Top 5 positive and negative keywords used
5) Sentiment sales co-relation index : Is online sentiment impacting sales yet ?
6) Competitive Sentiment Position : Where are we with respect to consumer sentiments on competition
7) Volume of discussion : More discussion means more buzz, positive or negative needs to be drilled down
8) Ratio of your entries with respect to competition :
9) Competitive sentiment ratio : Ratio of your sentiment index vis a vis competitive
10) Buzz velocity : Rate at which entries are coming up

The basic process of text mining consists of the following
Step-1 : Using a data extraction adaptor to pull entries which meet a certain criteria ( date , keyword ) from a URL
Step-2 : Indexing - Splitting the sentence into token words. Ex : "I liked the Dockers trousers" is broken down into "I" , "liked", "the" etc
Step-3 : Filter stop words - Words like "I", "the" are weeded out using a filtering process
Step-4 : Stemming . Words like 'analyze', 'analysis', etc are collapsed together into one keyword
Step-5 : Generate themes and identify co-relations among keywords, between keywords and products