Introducing

A User Centric News Article Application

BACKGROUND

News websites and applications are gradually being replaced by user-centric news applications. One of the major reasons behind this change is that these apps provide a personalised experience to the users where they can get their desired content without having to go through the long list of articles.

ARCHITECTURE

About Us

DATA PIPELINE


In a data pipeline, data is moved from one point (the source) to another (the destination). At each point, the source data is transformed and optimized, arriving in a form that can be analyzed and utilised for business purposes.



Data Sourcing
  • The New York Times Developer Network API allows to programmatically access New York Times data to extract information in a variety of ways.
  • To extract the relevant content from the article url, the html is parsed through a python library called BeautifulSoup.
  • Ensuring quality of data sourcing, Great Expectations reports enables checks and profiling of incoming data.
    • Data Processing
  • Pre-trained NLP models from Hugging Face are leveraged to summarize and translate article text. These summarization and translation are a key for users for making sense of large article.
  • Automate and orchestrate the execution of Python code on regular basis, Apache Airflow DAG are configured to run daily as data sourcing jobs.
  • Data are processed and stored in MongoDB, which provides the ability to build upon flexible schemas and operate much faster than a relational database.
    • Data as a Service
  • Create REST interface using Fast API to call commonly used functions to implement applications functionality of User Registration and Authentication, Retrieve article data from MongoDB, Record User Read activity into BigQuery, Send Article on IM.
  • Using Streamlit's simple framework to build interactive web application.

    • DEMO

    Try It Out

    Request Access
    CodeLab Doc

    Detailed Technical Documentation

    Client Image
    FastAPI Doc

    Public API User Documentation.

    Client Image
    Web App

    Try it out, Go ahead and enjoy reading.

    • ABOUT ME

    • I am grad student at Northeastern learning to build impactful software application

      News Aggregator
    • CONTACT INFO

    • Contact me for an issue or suggestion

      Boston, MA

      anand.pi@northeastern.edu

    Copyright © Piyush