Totango Engineering

Luigi and Spark in production at Totango

Totango transforms customer usage data into actionable insight and analytics. The pipeline execution is orchestrated by luigi open-source project by spotify, and the data transformation is done in Apache Spark.

Our pipeline runner makes sure the right compute jobs are being scheduled and processed so our users get accurate and timely information. The aforementioned technologies let us concentrate on the business logic instead of the plumbing.

The Big things are happening here Meetup group is devoted to promoting collaboration between Big Data developers in Israel, for working with the latest technologies, using the best practices. I have presented our architecture in the meetup, to introduce fellow developers to Luigi and our usage of it.

You can see the presentation slideshow itself below, or watch the presentation video (Hebrew)

Feel free to contact me for follow-up questions!

-- Romi Kuntsman, Big Data Engineer @ Totango ( LinkedIn / email )

( Note: Cross posted in Some Big Things blog )

Romi Kuntsman