A Study on the journey of Natural Language Processing models: from Symbolic Natural Language Processing to Bidirectional Encoder Representations from Transformers

Authors

  • Rajarshi SinhaRoy  Department of Computer Science, St. Xavier's College Kolkata, Kolkata, West Bengal, India

DOI:

https://doi.org/10.32628/CSEIT217688

Keywords:

Natural language Processing (NLP), Artificial Intelligence, Machine Learning, History of NLP, computational semantics, machine translation

Abstract

In this digital era, Natural language Processing is not just a computational process rather it is a way to communicate with machines as humanlike. It has been used in several fields from smart artificial assistants to health or emotion analyzers. Imagine a digital era without Natural language processing is something which we cannot even think of. In Natural language Processing, firstly it reads the information given and after that begins making sense of the information. After the data has been properly processed, the real steps are taken by the machine throwing some responses or completing the work. In this paper, I review the journey of natural language processing from the late 1940s to the present. This paper also contains several salient and most important works in this timeline which leads us to where we currently stand in this field. The review separates four eras in the history of Natural language Processing, each marked by a focus on machine translation, artificial intelligence impact, the adoption of a logico-grammatical style, and an attack on huge linguistic data. This paper helps to understand the historical aspects of Natural language processing and also inspires others to work and research in this domain.

References

  1. Schank, R. C., Conceptual Information Processing, Amsterdam, North Holland, 1975.
  2. Siemens AG (ed) Verbmobil: Mobiles Dolmetschgerät; Studie, Siemens AG, München, 1991.
  3. Sparck Jones, K. “Natural language processing: an overview”, International encyclopedia of linguistics (ed W. Bright), New York: Oxford University Press, Vol. 3, pp. 53-59, 1992.
  4. Sparck Jones, K. “Thesaurus”, Encyclopedia of artificial intelligence (ed Shapiro), 2nded, New York: Wiley, pp. 1605-1613, 1992.
  5. Walker, D.E. “SAFARI: an on-line text-processing system”, Proceedings of the American Documentation Institute Annual Meeting, pp. 144-147, 1967.
  6. Winograd, T. “A procedural model of language understanding”, pp. 249-266, 1973.
  7. Woods, W.A. “Semantics and quantification in natural language question answering”, pp. 205-248, 1978.
  8. Yngve, V.H. “MT at MIT”, pp. 451-523, 1967.
  9. Hutchins, W.J. Machine translation, Chichester, England: Ellis Horwood, 1986.
  10. Hutchins, W.J. and Somers, H.L. An introduction to machine translation, London: Academic Press, 1992.
  11. Jacobs, PS. (ed) Text-based intelligent systems, Hillsdale, NJ: Lawrence Erlbau Associates, 1992.
  12. Joshi, A.K., Webber, B.L. and Sag, I. A. (eds.) Elements of discourse understanding, Cambridge: Cambridge University Press, 1981.
  13. Kay, M., Gawron, J.M. and Norvig, P. Verbmobil: a translation system for faceto- face dialogue, CSLI, Stanford University, 1991.
  14. Kittredge, R. and Lehrberger, J. (eds.) Sublanguage: studies of language in restricted semantic domains, Berlin; Walter de Gruyter, 1982.
  15. Kobsa, A. and Wahlster, W. (eds.) User modelling in dialogue systems, Berlin: Springer-Verlag, 1989.
  16. Lea, W.A. (ed) Trends in speech recognition, Englewood Cliffs, NJ: Prentice-Hall, 1980.
  17. Locke, W.N. and Booth, A.D. (eds.) Machine translation of languages, New York: John Wiley, 1955.
  18. McKeown, K.R. Text generation, Cambridge: Cambridge University Press, 1985.
  19. Minsky, M. (ed.) Semantic information processing, Cambridge, 1968.
  20. Minsky, M., “A framework for representing knowledge,” (ed Winston, P.), The psychology of computer vision, McGraw-Hill, 1975.
  21. Nagao, M. (ed) A Japanese view of machine translation in light of the considerations and recommendations reported by ALPAC, USA, Japan Electronic Industry Development Association, 1989.
  22. Plath, W “Multiple path analysis and automatic translation”, in Booth, pp. 267-315, 1967.
  23. Reifler, E. “Chinese-English machine translation, its lexicographic and linguistic problems”, in Booth, pp. 317-428, 1967.
  24. Rumelhart, D.E., McClelland, J.L. and the POP Research Group, Parallel distributed processing, in Cambridge, vol. 2, 1986.
  25. Rustin, R. (ed) Natural language processing, New York: Algorithmics Press, 1973.
  26. HLT: Proceedings of the ARPA Workshop on Human Language Technology, March 1993; San Mateo, CA: Morgan Kaufmann, in press.
  27. Dahl D, “Natural Language Processing: Past, Present and Future”, Springer New York, pp. 49-73,  2013.
  28. Diksha Khurana, Aditya Koli, Kiran Khatter, Sukhdev Singh, "Natural Language Processing: State of The Art, Current Trends and Challenges"
  29. Nation, K., Snowling, M. J., & Clarke, "Dissecting the relationship between language skills and learning to read: Semantic and phonological contributions to new vocabulary learning in children with poor reading comprehension. Advances in Speech Language Pathology", pp. 131-139, 2007.
  30. Feldman, "NLP Meets the Jabberwocky: Natural Language Processing in Information Retrieval.", ONLINE-WESTON THEN WILTON-, vol. 23, pp. 62-73, 1999.
  31. https://www.analyticsvidhya.com/blog/2019/06/understanding-transformers-nlp-state-of-the-art-models/
  32. https://medium.com/analytics-vidhya/neural-machine-translations-implementing-encoder-decoder-658c3facd530
  33. Hutchins, W. J., "Early years in machine translation: memoirs and biographies of pioneers", Vol. 97, 2000. 
  34. Woods, W. A., "Semantics and quantification in natural language question answering.", vol. 17, pp. 1-87, 1978.
  35. Kamp, H., & Reyle, "Tense and Aspect. In From Discourse to Logic", Springer Netherlands, pp. 483-689, 1993.
  36. Eugene Charniak and Drew McDermott, Introduction to Artificial Intelligence , Pearson, 1998, Chapter 4.
  37. K.R. Chowdhary Professor & Head CSE Dept. M.B.M. Engineering College, Jodhpur, India. April 29, 2012 Natural Language Processing.
  38. M. Li, S.J. Liu, D.D. Zhang and M. Zhou, Machine Translation, Beijing:Higher Education Press, 2018.
  39. Y. Wang, "Natural language processing and applications in machine learning", Modern Chinese, vol. 5, pp. 187-191, 2019.
  40. Y.F. Song, "The development history and current situation of natural language processing", China High-Tech, vol. 3, pp. 64-66, 2019.
  41. M. Wang, S.W. Yu and X.F. Zhu, "Natural language processing and its applications in education", Mathematics in Practice and Theory, vol. 40, no. 20, pp. 151-156, 2015.
  42. K.B. Hu and Y. Li, "The features of machine translation and its relationship with human translation", Chinese Translators Journal, vol. 37, no. 5, pp. 10-14, 2016.
  43. Z.W. Feng, "Parallel development of machine translation and artificial intelligence", Journal of Foreign Languages, vol. 41, no. 6, pp. 35-48, 2018.
  44. Z.W. Feng, "Computational linguistics: its past and present", Journal of Foreign Languages, vol. 34, no. 1, pp. 9-17, 2011.
  45. Z.H. Zhou, Machine Learning, Beijing:Tsinghua University Press, 2016.
  46. W.J. Hutchins, Machine Translation: Past Present Future, Chichester:Ellis Horwood Limited, 1986.
  47. Rajarshi Sinharoy, Swarnendu Sarkhel, "Air Quality Index Prediction in Realtime Using SVM based model in Machine Learning", International Journal of Innovative Research in Physics, vol. 3, pp. 43-49, 2021.
  48. Q.P. Jiang, "Challenges and future of natural language processing", China Computer & Communication, vol. 14, pp. 219-221, 2013.
  49. B. Manaris, "Natural language processing: a human-computer interaction perspective", Advances in Computers, vol. 47, pp. 1-66, 1998.
  50. Y. Bar-Hillel, "The present status of automatic translation of languages", Advances in Computers, vol. 1, pp. 91-163, 1960.
  51. I. Sutskever, V. Oriol and V.L. Quoc, "Sequence to sequence learning with neural networks", Advances in Neural Information Processing Systems, vol. 4, pp. 3104-3112, 2014.
  52. P.E. Brown, J.D. Vincent, A.D. Stephen and L.M. Robert, "The mathematics of statistical machine translation: parameter estimation", Computational Linguistics, vol. 19, no. 2, pp. 263-311, 1993.
  53. Young, S. J., & Chase, "Speech recognition evaluation: a review of the US CSR and LVCSR programmes", pp. 263-279, 1998.  

IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology

Downloads

Published

2021-12-30

Issue

Volume 7, Issue 6, November-December-2021

Section

Research Articles

License

Copyright (c) IJSRCSEIT

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Rajarshi SinhaRoy, " A Study on the journey of Natural Language Processing models: from Symbolic Natural Language Processing to Bidirectional Encoder Representations from Transformers" International Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 7, Issue 6, pp.331-345, November-December-2021. Available at doi : https://doi.org/10.32628/CSEIT217688