Trends In Natural Language Processing : Scope And Challenges

Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to process and analyse large amounts of natural language data. The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them.NLP combines computational linguistics — rule-based modelling of human language — with statistical, machine learning, and deep learning models. Together, these technologies enable computers to process human language in the form of text or voice data and to ‘understand’ its full meaning, complete with the speaker or writer’s intent and sentiment. Chal lenges in natural language processing frequently involve speech recognition, natural language understanding, and natural language generation. This paper discusses on the various scope and challenges , current trends and future scopes of Natural Language Processing.


I. INTRODUCTION
As human beings started evolving and became social and organized animals, communication have played a huge role for mankind to come this far.With the advancement oftechnology, people recognized the importance of translation from one language to another and hoped to create a machine that could do this sort of translation automatically.
Natural Language processing (NLP) is the technology based on AI that enables the computers to understand human language whereas until some years earlier they were only capable of understanding mathematical language. [15].Some of important events in history of NLP: 1950 -NLP started with the publication of an article called "Machine and Intelligence" by Alan Turing. 2000-A large amount of spoken and textual data becоme available. [13] That is, the evolution of NLP included the following major milestones: Symbolic NLP (1950s-Early 1990s), Statistical NLP (1990s-2010s), Neural NLP (Present) [6].NLP is divided into two parts, i.e. Natural Language Understanding and Natural Language Generation, to make processing easier. [13] Linguistics is the scientific study of language. It invоlvesanalyzing language form, language meaning and language in context [14], Its study includes-Sounds which refers to phonology, Word formation refers to morphology, Sentence structure refers to syntax, Meaning refers to semantics, Understanding refers to pragmatics [4].
Noah Chomsky, one of the first linguists of twelfth century that started syntactic theories, marked a unique position in the field of theoretical linguistics [9].
Some of the common tasks of NLP include Speech recognition,Part of speech tagging, Natural language generation [1]. But, there are some limitations in NLPwhich include Contextual words and phrases and homonyms, Ambiguity, Errors in text or speech [5]. Natural Language Understanding:NLU is the process of conversion of unstructured data (data in human language) to structured data (form compatible for computer operations). [13] Its task is to understand and reason where input is a natural language. [4] Natural Language Generation: NLG is the technology used by computers to communicate with humans in a way that they can understand. [13]It is a sub generation of natural language processing. It is also referred to as text generation. [4]  • Morphological and Lexical AnalysisThe lexicon of a language is its vocabulary that includes its words and expressions. Morphology depicts analysing, identifying and description of structure of words. [4] Lexical Analysis-The process of splitting a sentence into words or small units called "tokens" in order to identify the meaning of it and its relationship to the entire sentence. [7] • Syntactic Analysis This involves analysation of the words in a sentence to depict the grammatical structure of the sentence. The words are transformed into structure that shows how the words are related to each other Eg. "the girl the go to the school". This would definitely be rejected by the English syntactic analyser. [4] • Semantic Analysis This abstracts the dictionary meaning or the exact meaning from context. The structures which are created by the syntactic analyser are assigned meaning.Eg.
"colourless blue idea".This would be rejected by the analyser as colourless blue do not make any sense altogether. [4] Output Transformation:Theprocess of generating an output based on the semantic analysis of the text or speech which fits the target of the application. [7] • Discourse Integration The meaning of any single sentence depends upon the sentences that preceeds it and also invokes the meaning of the sentences that follow it .Eg the word "it" in the sentence "she wanted it" depends upon the prior discourse context. [4] • Pragmatic Analysis It means abstracting or deriving the purposeful use of the language in  So for example, a fruit may be considered to be an apple if it is red, round, and about 3" in diameter. A Naive Bayes classifier considers each of these "features" (red, round, 3" in diameter) to contribute independently to the probability that the fruit is an apple, regardless of any correlations between features. Features, however, aren't always independent which is often seen as a shortcoming of the Naive Bayes algorithm and this is why it's labeled "naive". Although it's a relatively simple idea, Naive Bayes can often outperform other • Part of speech tagging, also called grammatical tagging, is the process of determining the part of speech of a particular word or piece of text based on its use and context. Part of speech identifies 'make' as a verb in 'I can make a paper plane,' and as a noun in 'What make of car do you own?' [1] • Word sense disambiguation is the selection of the meaning of a word with multiplemeaningthrough a process of semantic analysis that determine the word that makes the most sense in the given context. For example, word sense disambiguation helps distinguish the meaning of the verb 'make' in 'make the grade' (achieve) vs. 'make a bet' (place  • NLP in Recruitment: NLP can also be used in both search and selection phases of Job Recruitment, in fact, the chatbot can also be used to handle the job-related query at Initial level which also includes identifying the required skills for a specific job and handling initial level tests and exams. [2] VIII. LIMITATIONS AND CHALLENGES OF NLP There are a number of challenges of natural language processing and most of them boil down to the fact that natural language is ever-evolving and always somewhat ambiguous. They include: • Errors in text and speech Misspelled or misused words can create problems for text analysis.
Autocorrect and grammar correction applications can handle common mistakes, but don't always understand the writer's intention.
With spoken language, mispronunciations, different accents, stutters, etc., can be difficult for a machine to understand. [5] • Contextual words and phrases and homonyms. every day, many of these barriers will be broken through in the coming years.