Using PostgreSQL for Machine Learning Data Pipelines

From: Pankaj Jangid <pankaj(dot)jangid(at)gmail(dot)com>
To: Postgres General <pgsql-general(at)postgresql(dot)org>
Subject: Using PostgreSQL for Machine Learning Data Pipelines
Date: 2019-10-17 18:04:43
Message-ID: m236frw8n8.fsf@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

I am working on a machine-learning project. Because of the available
study material in the ML area, the team is inclined towards Apache
Kafka, Apache Spark for data-pipelines and analytics.

Our requirement is to store huge amounts of continuously increasing data
that cannot fit into a single machine. The algorithms require data in
batches so it is not necessary to keep full data ready for
consumption. Using Kafka, the data can be distributed and fetched in
varying batch sizes as and when required.

I am more comfortable with PostgreSQL. And wanted to know more about
case-studies where PostgreSQL is deployed for ML use. Any pointers
referring to study material will be helpful. Please share in this
thread.

--
Thanks & Regards,
Pankaj Jangid

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Michael Lewis 2019-10-17 18:21:18 Re: Vacuum very big table - how the full vacuum works in background/internally?
Previous Message Julie Nishimura 2019-10-17 17:20:09 releasing space