We’re Hiring an Apache Spark Developer
This role is responsible for building and maintaining the architecture and systems for automating predictive models and the components used to train, predict, and track the accuracy of them. This includes gathering, structuring and analyzing data in high volumes and velocity.
Collect, store and manage large data sets from multiple source systems and third parties.
Design, build, and continuously improve the architecture and systems to support predictive models in a distributed environment.
Create processes to monitor the prediction systems to ensure they are performant and produce accurate predictions.
Collaborate with data science team to support any data needs and understand model structures to be automated by the system.
Work closely with data science and analytics teams to identify correct data segmentation, best algorithm to apply, and which features to use in the model.
Anticipate future demands of initiatives related to people, technology, and business when designing solutions.
Collaborate with Business Intelligence and other IT teams when designing processes to persist model predictions, calling models from other systems via API, and providing visuals for reporting data insights to the business.
At least 1 years of hands on experience in Spark (fundamentals, RDD and dataframe APIs)
At least 2 years of experience with Python ML stack (scikit-learn, pandas, numpy) Experience with SparkML (or any ML on a big data platform)
At least 2 years of experience in Hadoop and knowledge of HDFS design patterns
Strong understanding of machine learning theory (supervised and unsupervised algorithms, regularization, cross-validation, hyper-parameter grid search, predictive performance evaluation, optimization concepts)
Advanced SQL Skills (multiple table queries, subqueries, window functions)
Ability to quickly adjust to a growing organization’s needs, new markets, and opportunities in a collaborative Agile environment Team oriented with focus on collaboration, knowledge sharing, and relationship building High degree of responsiveness, reliability, and integrity Decisiveness, good judgment, analytical, and problem-solving skills
C# programming Experience with neural networks and streaming data Experience with Azure data stack Background in statistics
“The people I am surrounded by are simply the best, and I am proud to call them friends.”
― Lori Davis, Technical Recruiter