Data Engineering
One of the biggest information service providers in the UK
Goal of the project was to identify commodities and quantities from bills of lading using text mining, build integrations from the Warehouse as well as from webpages onto a cloud data processing model. The historic data fetched was used by the model to determine the predicted price based on the source and destination of shipping, nature and quantity of commodity being shipped. Current pricing of commodities needed to be harvested from webpages and pushed onto the model repository.
The handwritten bills of lading are used to determine the commodity and quantity associated. Improper/incomplete information on these forms limit the solution for such cases. Currency and unit measure would vary based on the data from these forms and these are primary factors determining the fair price. When harvesting price data of commodity, the common products in each category are considered for pricing input onto the data lake.
For a detailed presentation of specific use cases, please write back to us at support@techmobius.com
Goal of the project was to identify commodities and quantities from bills of lading using text mining, build integrations from the Warehouse as well as from webpages onto a cloud data processing model. The historic data fetched was used by the model to determine the predicted price based on the source and destination of shipping, nature and quantity of commodity being shipped. Current pricing of commodities needed to be harvested from webpages and pushed onto the model repository.
The handwritten bills of lading are used to determine the commodity and quantity associated. Improper/incomplete information on these forms limit the solution for such cases. Currency and unit measure would vary based on the data from these forms and these are primary factors determining the fair price. When harvesting price data of commodity, the common products in each category are considered for pricing input onto the data lake.
An end-to-end automated solution employing Text engineering & Machine learning models, data integration from Cloud data lake, data harvesting and ETL and a web component for display of the same.
Customized user interface has been developed and deployed for the customer to query the price records and view the results.
Decoupled each component and enabled API service for integrating each of the architecture components with any system.
A generic component has been built to handle currency conversion, unit conversions and the price calculations (mean and average).
The solution we have provided is a highly scalable model. We can add or change the data repositories easily.
The solution will produce following insights based on the price records available on the data repository:
Entire solution is implemented on cloud infra with automated service availability checks to make sure the service is up and running all the time.
All the components are de-coupled and the communication between components happens through API calls. So altering the workflow/architecture will be simple.
Price Monitoring Tool Price monitoring tool DevOps Client A leading Australian market place…
Reporting Tool Solution Reporting tool solution Data Consulting Client A Renowned software service…
Smart Product Classification Smart Product Classification Website Development Client A leading US-based online…
