OperationsFramework / modelModelAccessible

Text analytics

How can text analytics improve people, teams, or organisational effectiveness?

AccessibleOperationalIndividual3 min read

Contents

Text analytics, also known as text mining, is a process of extracting value from large quantities of unstructured text data.

Text analytics, or text mining, uses computational methods to extract useful structure, patterns and signals from collections of unstructured language.

When to use it

Five common tasks define much of the field:

Text categorisation assigns documents to known classes, such as topic, document type or author. It supports spam detection, email routing and searchable archives.

Text clustering discovers groups without requiring predefined labels. A search for ‘cell’, for example, may separate results about biology, batteries and prisons.

Concept extraction identifies entities, topics and relationships. In legal discovery, it can narrow millions of documents to the material most likely to matter.

Sentiment analysis, also called opinion mining, estimates whether language expresses positive, negative or neutral attitudes and helps reveal patterns beyond literal wording.

Document summarisation produces a shorter representation of the important content, helping readers triage large volumes of material.

Together, these methods support retrieval, tagging, pattern recognition, information extraction and predictive work. Useful business questions include:

What do customers or employees think about a product? See Sentiment Analysis.

How is the employment brand perceived in public conversation?

Which complaints recur most often?

What themes are emerging in on-site search behaviour?

Origins

Text analytics has no single inventor. It developed from information retrieval, computational linguistics, statistics and natural-language processing. Early systems focused on indexing and word frequency; later machine-learning methods made classification, clustering and sentiment analysis practical at scale. Contemporary systems combine linguistic rules, statistical models and, increasingly, learned language representations.

What it is

Businesses hold large text collections in documents, email, reports, customer records, websites, blogs and social channels. Humans can read those materials, but conventional databases cannot analyse prose as neatly as rows and columns. Traditional metadata—file name, author and creation date—helps retrieve a known document, and keyword search finds a known term. Neither reliably discovers an unanticipated pattern.

Text analytics converts language into analysable features so a system can surface change, association and anomaly. It might detect a decline in customer sentiment, reveal an emerging product request or connect issues that were previously scattered across thousands of records. The output is evidence for interpretation, not an automatic statement of truth: irony, domain language, multilingual text and biased training data can all affect results.

How to use it

Start with a decision and a measurable question. Identify the relevant corpus, permissions and retention rules before processing anything. Text must be datafied, not merely digitised: a photograph of a page is digital, but its words are not machine-readable until optical character recognition or transcription converts them into text.

The scale of digitisation illustrates the distinction. Estimates suggest that more than 130 million books have appeared since Gutenberg’s press in 1450; by 2012, Google’s book project had scanned over 20 million titles, around 15 per cent of the written heritage described in the original estimate. A scan preserves the page image, while datafied text supports search, resizing, annotation and analysis.

Prepare the corpus by correcting encoding and recognition errors, removing duplicates, retaining useful metadata and documenting any exclusions. Depending on the task, common ‘stop words’ may be removed, but do not do this automatically: small function words can matter for sentiment, authorship and meaning. Choose the technique that matches the question, create a labelled validation sample and test performance on the language and population in scope.

Interpret the result with domain experts. Inspect false positives and false negatives, assess whether patterns hold across groups and time periods, and avoid converting uncertain model output into individual-level judgement. Commercial tools can accelerate the workflow, but they do not replace governance or validation.

Practical example

An employee-engagement survey may combine rating questions with open comments. Reading a sample can produce anecdotes, but text analytics can classify themes and compare their prevalence across divisions or teams. A word cloud may offer a quick visual cue, yet frequency alone does not establish importance or sentiment; theme coding, context and representative quotations provide stronger evidence.

Employee email or social content should never be treated as a convenient substitute for a survey without a lawful basis, transparent governance and proportionate safeguards. Workplace monitoring creates serious privacy, consent and power concerns, and a technically possible analysis may still be inappropriate.

Top practical tip

Define the decision and evaluation criteria before choosing a text-analytics technique or collecting a corpus.

Top pitfall

Do not digitise an entire archive simply because it exists. Confirm relevance, rights, expected value and the cost of making the text analysable.