The organisation that you selected had collected free-format customer feedbacks over the years and will like to do a text mining project to understand their customers better. Outline five (5) challenges that the organisation is likely to face with the textual data and suggest how they can be overcome.
Expert Answer
Text mining is the process of analyzing textual information in order to know consumer preferences and needs. It is used to identify customer purchase patterns and opinions on particular products. Large amount of data is gathered everyday which needs to be organized to generate value out of it. The challenges faced by companies in text mining are as follows:
- The first challenge is to conduct a semantic analysis of to conclude a significant large amount of data in order to establish the meaningful relationship between the concepts. So the main challenge is to conduct semantic analysis in an efficient way.
- The documents consist of multilingual text whose refining is a major challenge because most of the text mining methods focus on English documents.
- It is challenge that how a user’s knowledge is utilized to efficiently analyze the discovered knowledge and make it more interpretable.
- There may be word ambiguity and context sensitivity. for example Apple (the company) and apple (the fruit)
- There exist a complex and subtle relationship between concepts in text which needs to be taken care of in knowledgeable pattern.
There is a lack of widely-deployed techniques for automated analysis of unstructured text in the real world. In order to overcome these problems future text mining tools should express natural language queries and automatically implement the proper mining process without much complexity