Parse.ly Code and Tech
Parse.ly is a distributed team of Pythonistas and JavaScript hackers who aim to build the best real-time web analytics system in the world.
We are committed to the open source community and involve ourselves in the creation and maintenance of a number of projects, both core and experimental.
2009 - Present
We LOVE open source
Parse.ly makes use of a lot of open source technology. We keep a relatively up-to-date table of primary open source components used in our software services, along with their licenses.
We are currently working on a few open source projects, including tools to integrate Apache Kafka and Apache Storm with Python.
If you love open source, too, and want to work with cutting-edge open source technology and even release/maintain open source work as part of your day job, check out our software engineer job openings!
Core Projects
pykafka
Apache Kafka client library for Python
View it on githubstreamparse
Easy streaming computation with Python and Apache Storm
View it on githubpystorm
Battle-tested Apache Storm Multi-Lang implementation for Python
View it on githubbirding
Stream processing of Twitter's APIs with Python
View it on githubtestinstances
Managed test instances for integration tests
View it on githubnewspaper
News, full-text, article metadata extraction
View it on githubtime-engaged
Open source implementation of engaged time in JavaScript
View it on githubExperimental Projects
python-pds
Probabilistic data structures with Python
View it on githubreds
Enhanced Python data structures with Redis
View it on githubsolrcloudpy
Apache Solr + SolrCloud in Python
View it on githubschemato
Metadata validation and distilling with RDF
View it on githubdomshot
Convert d3.js into images with Jinja/PhantomJS
View it on githubemailipy
Library for inlining CSS into HTML for email
View it on githubserpextract
Extract search engine keywords from logs
View it on githubpminifier
bit.ly-style minifier with Redis/Mongo
View it on githubConference Presentations
Our team has presented at PyCon, PyData, and other major industry conferences.
- The Hidden Costs of On-Call: False Alarms LISA 2017 video
- Realtime Distributed Computing At Scale: Storm And Streamparse EuroPython 2017 video
- Beating Python's GIL to Max Out Your CPUs PyData NYC 2015 video slides
- Simplifying large scale parallel processing with streamparse PyData NYC 2015 video
- streamparse: real-time streams with Python PyCon US 2015 video slides notes
- Real-time streams w/ Kafka & Storm PyData SV 2014 video slides notes
- Rapid Data Visualization PyData NYC 2013 video slides
- Probabilistic Data Structures PyData NYC 2012 video code
- Web Crawling & Metadata Extraction PyData NYC 2012 video
- NLTK & Text Processing PyData NYC 2012 video
- Wikipedia Indexing & Analysis PyData NYC 2012 video
- Parse.ly Lightning Talk PyData NYC 2012 video
- Rapid Web Development w/ Lightweight Tools PyCon 2012 video