Quick Learnology

What is NLP ?

Natural language processing (NLP) can be defined as the automatic (or semi-automatic) processing of human language.

NLP is essentially multidisciplinary: it is closely related to linguistics, ML , Human Computer Interaction and AI.
Natural language processing is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data.

NLP: related to the area of Human computer Interaction.

 Ex:

  • Paani ,pehla
  •  Paanee ,Pahla
  •  Paanie ,Pahlaa
  •  Paanii, Pehlaa etc…

Text Analysis or Text Processing:

Process of gleaning high quality and meaningful information ( through devising of patterns and trends by means of statistical pattern learning) from text.

Ex:

  • Text categorization- H,E,O etc….
  • Text clustering
  • Sentiment analysis– joy,sad,anger,hatred etc..
  • Content/Intent extraction

Some linguistic terminology :

 

Morphology:

In linguistics, morphology is the study of words, how they are formed, and their relationship to other words in the same language. It analyzes the structure of words and parts of words such as stems, root words, prefixes, and suffixes.

What is a morpheme example?

A morpheme is the smallest linguistic part of a word that can have a meaning. In other words, it is the smallest meaningful part of a word. Examples of morphemes would be the parts “un-“, “break”, and “-able” in the word “unbreakable”.

Morphemes :

Morphemes are the smallest units in a language that have meaning. They can be classified as free morphemes, which can stand alone as words, or bound morphemes, which must be combined with another morpheme to form a complete word.
Bound morphemes typically appear as affixes in the English language.

  • Free and Bound Morphemes
  • There are two types of morphemes-free morphemes and bound morphemes. “Free morphemes” can stand alone with a specific meaning, for example, eat, date, weak.
  • Bound morphemes” cannot stand alone with meaning.
  • An example of a “bound base” morpheme is -sent in the word dissent.

Phoneme :

  • In phonology and linguistics, a phoneme is a unit of sound that can distinguish one word from another in
    a particular language.
  • Phoneme, in linguistics, smallest unit of speech distinguishing one word (or word element) from another, as the element p in “tap,” which separates that word from “tab,” “tag,” and “tan.”

Grapheme:

  • A Grapheme is a symbol used to identify a phoneme; it’s a letter or group of letters representing the sound. You use the letter names to identify Graphemes, like the “c” in car where the hard “c” sound is represented by the letter “c.” A two-letter Grapheme is in “team” where the “ea” makes a long “ee” sound.
  • In linguistics, a grapheme is the smallest functional unit of a writing system.

Hard Parts of NLP :

One word can have a zillion(extremely large) different semantic meanings

  •  Consider: Take
  • “take a place at the table”
  • “take money to the bank”
  • “take a picture”
  •  “take a lot of time”
  • “take Medicine”

We can use one word in different sentences.

ONE BIG QUESTION ARISES IN OUR MIND :

Why is language processing difficult?

Natural language processing is considered a difficult problem in computer science. What makes NLP difficult is the nature of human language.

The rules governing the communication of information in natural language are not easily understood by computers.

Some of these rules may be high-level and abstract. For example, when someone makes a sarcastic remark to convey information.

On the other hand, some of these rules may be low level. For example, use the ‘s’ letter to indicate multiple elements.

A complete understanding of human language requires understanding both the context of words and concepts to convey the intended message.

Humans can easily learn a language, but the ambiguity and imprecise properties of natural language make it difficult for machines to implement NLP.

In simple words , we use spelling of Quick Learnology and another one use quick learnology and third person use quik lernology three persons use in different style and three are right but way of writing is different thats why NLP is tough because we have to train the model again and again in different ways.

NLP APPLICATIONS :

  • Spelling and Grammar Checking
  • Optical Character Recognition
  • Screen readers for blind and partially sighted users
  • Document Classification 
  • Document Clustering
  • Information Extraction
  • Text Summerization
  • Text Segmentaion
  • Exam Marking
  • Sentimental Analysis
  • Chatbots and Virtual Assistance
  • E-mail Understanding
  • Speech Recognition and many more applications 

What is Different about NLP from the rest of Computer Science ?

Most algorithms in computer science have a “right” answer:
Consider the two problems:
 – Sort the following ten integers
 – Find the highest integer among given integers
Now consider:
 – Find the document most relevant to “Apple as a Company”

If you don’t get the “correct” answer, the algorithm is considered flawed.
 Heuristics try to guess something close to the correct answer.
Heuristics are measured by “how close” to the correct answer.
 Information extraction by NLP methods is fundamentally heuristic because the correct answer is unknown.
 Now we need to measure how close we can get to the correct answer

Federated Search :

  • An information retrieval technology that allows you to search multiple searchable resources simultaneously.
  • A user submits a single query that is distributed to the search engines participating in the group.
  • Federated Search then aggregates the results it receives from the search engine and presents them to the user