A PROPOSED DISTRIBUTED ARCHITECTURE FOR SEARCHING SEMANTICALLY ON A LARGE DATASET OF HACKING NEWS

Authors

  • Ngoc Long Do Command 86
  • The Hung Nguyen Command 86
  • Trung Dung Nguyen Command 86
  • Xuan Duc Le Command 86
  • Chi Thanh Nguyen Academy of Military Science and Technology
  • Quoc Khanh Nguyen Le Quy Don Technical University
  • Thi Bich Van Pham Le Quy Don Technical University

DOI:

https://doi.org/10.56651/lqdtu.jst.v14.n01.1040.ict

Keywords:

Distributed system, knowledge graph, semantic search, natural language processing, hackernews, large language models

Abstract

In this paper, we propose a distributed architecture to support semantic search on largescale datasets of online technology news. The solution combines knowledge graph modeling, natural language processing techniques, and distributed processing on Apache Spark. The paper presents: (1) A distributed architecture that stores a large-scale semantic news dataset using resource description framework model; (2) A pipeline for extracting knowledge from text using natural language processing (NLP) tools such as dependency parsing and named entity recognition; (3) A distributed search engine that uses keyword expansion and graph reasoning to return semantically related results. The experimental results show that the proposed model improves the semantic search capabilities on large-scale data compared to traditional keywordbased search methods.

Downloads

Published

2025-07-04

Issue

Vol. 14 No. 01 (2025)

Section

Articles