Authors : Arnab Sen

Volume/Issue : Volume 10 - 2025, Issue 11 - November

Google Scholar : https://tinyurl.com/6xcdsuza

Scribd : https://tinyurl.com/3bn5fc4

DOI : https://doi.org/10.38124/ijisrt/25nov395

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 30 to 40 days to display the article.

Abstract : Background: This research addresses the fundamental trade-off between model complexity and operational efficiency in Natural Language Processing (NLP), specifically for resource-constrained environments like edge computing.1 While Large Language Models (LLMs) offer unprecedented capabilities, their massive resource demands necessitate efficient alternatives.1 Materials and Methods: A critical comparative analysis was conducted on two dominant language model architectures: statistical N-gram models and modern Transformer-based Small Language Models (SLMs).1 The study evaluates their architectural mechanisms, efficiency metrics, tokenization strategies, and performance trade-offs, particularly focusing on metrics such as Perplexity (PPL) and qualitative semantic coherence.1 Results: SLMs, leveraging architectural optimizations like knowledge distillation and quantization, provide superior contextual understanding and deployment efficiency (days/weeks of training on small clusters) over N-gram models.1 N-gram models are severely limited by data sparsity, finite context windows, and storage bottlenecks, despite their fast lookup times.1 SLMs' use of subword tokenization (BPE) effectively eliminates the Out-of-Vocabulary (OOV) problem, preserving information lost by the N- gram’s generic $\langle \text{unk} \rangle$ token.1 Conclusion: Resource-optimized SLMs are the most effective solution for high-performance, specialized NLP tasks in edge computing.1 While N-grams retain a niche as high-precision baselines for purely local statistical distributions, the efficiency and depth of comprehension favor the SLM for modern applications.1

Keywords : Edge Computing; N-gram Models; Perplexity; Small Language Models; Transformer.

References :

1. Plagiarism Free Writing Techniques: Avoiding Common Pitfalls in Research Writing - San Francisco Edit, accessed on November 7, 2025, https://www.sfedit.net/plagiarism-free-writing-techniques-avoiding-common-pitfalls-in-research-writing/
2. How to Write a Plagiarism-Free Research Paper or Thesis - Papergen AI, accessed on November 7, 2025, https://www.papergen.ai/blog/how-to-write-a-plagiarism-free-research-paper-or-thesis
3. How to Avoid Plagiarism | Harvard Guide to Using Sources, accessed on November 7, 2025, https://usingsources.fas.harvard.edu/how-avoid-plagiarism-0
4. Best Practices to Avoid Plagiarism - Purdue OWL, accessed on November 7, 2025, https://owl.purdue.edu/owl/avoiding_plagiarism/best_practices.html
5. IOSR Manuscript Preparation Guidelines | PDF - Scribd, accessed on November 7, 2025, https://www.scribd.com/document/768600584/IOSR-Manuscript-Preparation-Guidelines
6. Paper preparation guidelines for IOSR Journal of Engineering, accessed on November 7, 2025, https://ternaengg.ac.in/equinox2018/PaperFormat.pdf
7. Manuscript Preparation Guidelines (2 Page) | PDF | Abstract (Summary) | Paragraph - Scribd, accessed on November 7, 2025, https://www.scribd.com/document/98842041/Manuscript-Preparation-Guidelines-2-Page
8. IOSR Journal of Computer Engineering (IOSR-JCE) Template - International Organization of Scientific Research - SciSpace, accessed on November 7, 2025, https://scispace.com/formats/international-organization-of-scientific-research/iosr-journal-of-computer-engineering-iosr-jce/489e0da8074e4cfc8b861a6709e6969f
9. Paper Template - IOSR Journal, accessed on November 7, 2025, https://www.iosrjournals.org/doc/Paper%20Template.doc
10. N-gram Language Models - Stanford University, accessed on November 7, 2025, https://web.stanford.edu/~jurafsky/slp3/3.pdf
11. Word n-gram language model - Wikipedia, accessed on November 7, 2025, https://en.wikipedia.org/wiki/Word_n-gram_language_model
12. Transformer (deep learning architecture) - Wikipedia, accessed on November 7, 2025, https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)
13. Small Language Models (SLM): A Comprehensive Overview - Hugging Face, accessed on November 7, 2025, https://huggingface.co/blog/jjokah/small-language-model
14. SLM vs LLM: The Key Differences - WEKA, accessed on November 7, 2025, https://www.weka.io/learn/ai-ml/slm-vs-llm/
15. What Are Small Language Models (SLMs)? A Practical Guide - Aisera, accessed on November 7, 2025, https://aisera.com/blog/small-language-models/
16. Large and small language models: A side-by-side comparison - Rabiloo, accessed on November 7, 2025, https://rabiloo.com/blog/large-and-small-language-models-a-side-by-side-comparison
17. Understanding Language Modeling: From N-grams to Transformer-based Neural Models | by Roshmita Dey | Medium, accessed on November 7, 2025, https://medium.com/@roshmitadey/understanding-language-modeling-from-n-grams-to-transformer-based-neural-models-d2bdf1532c6d
18. LLM Transformer Model Visually Explained - Polo Club of Data Science, accessed on November 7, 2025, https://poloclub.github.io/transformer-explainer/
19. Comparing the Effect of Smoothing and N-gram Order - Scholarship Repository @ Florida Tech, accessed on November 7, 2025, https://repository.fit.edu/cgi/viewcontent.cgi?article=1712&context=etd
20. Faster and Smaller N-Gram Language Models - ACL Anthology, accessed on November 7, 2025, https://aclanthology.org/P11-1027.pdf
21. Faster and Smaller N-Gram Language Models - The Berkeley NLP Group, accessed on November 7, 2025, http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf
22. Summary of the tokenizers - Hugging Face, accessed on November 7, 2025, https://huggingface.co/docs/transformers/en/tokenizer_summary
23. Predictive Incremental Parsing Helps Language Modeling - ACL Anthology, accessed on November 7, 2025, https://aclanthology.org/C16-1026.pdf
24. Byte Pair Encoding vs. Unigram Tokenization: A Deep Dive into Subword Models - Medium, accessed on November 7, 2025, https://medium.com/@hexiangnan/byte-pair-encoding-vs-unigram-tokenization-a-deep-dive-into-subword-models-4963246e9a34
25. Two minutes NLP — A Taxonomy of Tokenization Methods | by Fabio Chiusano - Medium, accessed on November 7, 2025, https://medium.com/nlplanet/two-minutes-nlp-a-taxonomy-of-tokenization-methods-60e330aacad3
26. Arnab Sen Paper.docx
27. Can Transformers Learn n-gram Language Models? - ACL Anthology, accessed on November 7, 2025, https://aclanthology.org/2024.emnlp-main.550.pdf
28. A Comparison of Tokenization Impact in Attention Based and State Space Genomic Language Models | bioRxiv, accessed on November 7, 2025, https://www.biorxiv.org/content/10.1101/2024.09.09.612081v2.full-text
29. A Comparative analysis of different LLM Evaluation Metrics | by Satyadeep Behera - Medium, accessed on November 7, 2025, https://medium.com/@satyadeepbehera/a-comparative-analysis-of-different-llm-evaluation-metrics-98395c3d8e79
30. Perplexity Metric for LLM Evaluation - Analytics Vidhya, accessed on November 7, 2025, https://www.analyticsvidhya.com/blog/2025/04/perplexity-metric-for-llm-evaluation/
31. How to evaluate a text generation model: strengths and limitations of popular evaluation metrics - The Analytics Lab, accessed on November 7, 2025, https://theanalyticslab.nl/how-to-evaluate-a-text-generation-model-strengths-and-limitations-of-popular-evaluation-metrics/
32. LLM Evaluation: 15 Metrics You Need to Know, accessed on November 7, 2025, https://arya.ai/blog/llm-evaluation-metrics
33. Testing & Evaluating Large Language Models(LLMs): Key Metrics and Best Practices Part-2, accessed on November 7, 2025, https://medium.com/@sumit.somanchd/testing-evaluating-large-language-models-llms-key-metrics-and-best-practices-part-2-0ac7092c9776
34. Small Language Models: A Business Leader's Guide to Affordable, Task-Tuned AI, accessed on November 7, 2025, https://deliveringdataanalytics.com/small-language-models-business-guide/
35. The Rise of Small Language Models - IEEE Computer Society, accessed on November 7, 2025, https://www.computer.org/csdl/magazine/ex/2025/01/10897262/24uGPS4TUQ0