Parse.ly Code and Tech
We are committed to the open source community and involve ourselves in the creation and maintenance of a number of projects, both core and experimental.
2009 - Present
We LOVE open source
Parse.ly makes use of a lot of open source technology. We keep a relatively up-to-date table of primary open source components used in our software services, along with their licenses.
We are currently working on a few open source projects, including tools to integrate Apache Kafka and Apache Storm with Python.
If you love open source, too, and want to work with cutting-edge open source technology and even release/maintain open source work as part of your day job, check out our software engineer job openings!
Apache Kafka client library for PythonView it on github
Easy streaming computation with Python and Apache StormView it on github
Battle-tested Apache Storm Multi-Lang implementation for PythonView it on github
Stream processing of Twitter's APIs with PythonView it on github
Managed test instances for integration testsView it on github
News, full-text, article metadata extractionView it on github
Probabilistic data structures with PythonView it on github
Enhanced Python data structures with RedisView it on github
Apache Solr + SolrCloud in PythonView it on github
Metadata validation and distilling with RDFView it on github
Convert d3.js into images with Jinja/PhantomJSView it on github
Library for inlining CSS into HTML for emailView it on github
Extract search engine keywords from logsView it on github
bit.ly-style minifier with Redis/MongoView it on github
Our team has presented at PyCon, PyData, and other major industry conferences.
- The Hidden Costs of On-Call: False Alarms LISA 2017 video
- Realtime Distributed Computing At Scale: Storm And Streamparse EuroPython 2017 video
- Beating Python's GIL to Max Out Your CPUs PyData NYC 2015 video slides
- Simplifying large scale parallel processing with streamparse PyData NYC 2015 video
- streamparse: real-time streams with Python PyCon US 2015 video slides notes
- Real-time streams w/ Kafka & Storm PyData SV 2014 video slides notes
- Rapid Data Visualization PyData NYC 2013 video slides
- Probabilistic Data Structures PyData NYC 2012 video code
- Web Crawling & Metadata Extraction PyData NYC 2012 video
- NLTK & Text Processing PyData NYC 2012 video
- Wikipedia Indexing & Analysis PyData NYC 2012 video
- Parse.ly Lightning Talk PyData NYC 2012 video
- Rapid Web Development w/ Lightweight Tools PyCon 2012 video