AI-based SAS Product for Enterprises
Background
Vector, an Artificial Intelligence startup in Houston, approached us to develop a google type search engine for enterprises that can allow their Researchers and Data scientist to perform sentiments and sensitivities analysis towards certain topics discussed on news media and social media websites and also provided AI and Machine learning driven data and BI reports useful for data scientist to make predictions. This project was intended to be sold as subscription to enterprises across the US.
Objectives
App Maisters BI and AI team sat with the client after detail discovery session delivered a SaaS based application that provided quantitative research driven industries named- entity recognition and information extract of news media and social media and provided a search engine similar to google. Following were some of the deliverables.
- Subscription based website with yearly and monthly options
- Scrapper that scraps millions of pages and dumps into a Big Data repository that stores millions of websites and social media sites extract.
- Crunch huge amount of data using distributive computing.
- Use different algorithms to generate sensitivity score, sentiments, named entities and other useful information
- Search engine that allows for high speed search based on several criteria’s like keywords within couple of seconds.
- BI reporting that shows different stats and graphs
- Use k-means clustering and LDA topic extraction to schedule and generate downloadable files for further analysis.
Technology Stack
Backend: AWS EC2, Load balancer, CloudWatch, Amazon elasticsearch(Big Data), Amazon Apache EMR/Spark(distributed computing), AWS S3.
Languages: PHP, Python, JavaScript, MySQL, JSON
Third Party APIs : , SciKit, Scipy, PySpark, Stripe, JSON to CSV, CSV to JSON, Pandas
- Clients:
- Category:
BI & Analytics - Date: