Senior Data Engineer – Hux

What is Hux? Hux is the Human Experience Platform by Deloitte Digital.

In today’s world, customers expect companies to know who they are and what they want. Customers want to have products, services or experiences that best suit their needs delivered to them seamlessly across physical and digital channels.

Customers are human first: driven by dynamic wants, needs, and desires. The ability for brands to make personal, meaningful connections on a human level has never been greater and Hux by Deloitte Digital delivers on those experiences in a way that allows companies to own the customer journey end to end. We help companies connect key data sources to understand what matters most to people; connect to advanced technologies like AI and machine learning to sense and respond to those needs at scale; and connect their systems to unlock insights, create collaboration and drive acquisition, engagement and loyalty. Most importantly, we empower companies to connect with customers in personal, meaningful ways that respect them as people, not just customers.

Hux by Deloitte Digital gives companies the ability to build and leverage the connections – between people, systems, data and technologies – so they can deliver personalized, contextual experiences to customers at scale.

Work you’ll do

As a Senior Data Engineer-Hux, you’ll design, implement and maintain a full suite of real-time and batch jobs that fuels our cutting-edge AI to provide real-time marketing intelligence to our existing clients.

You’ll develop, test and deliver production-grade code to help our clients solve their most critical marketing challenges using cutting-edge big-data tools. You’ll also ensure data integrity, resolve production issues, and assist in the support and maintenance of our overall platform.

As you grow your capabilities and learn how to build a platform that can ingest, load and process billions of data points, you’ll enjoy new challenges and opportunities to showcase your development skills by joining project teams to build innovative new-client platforms and execute high-value strategic development projects with high visibility.

Your responsibilities will include:

Design, construct, install, test and maintain highly scalable data pipelines with state-of-the-art monitoring and logging practices.
Bring together large, complex and sparse data sets to meet functional and non-functional business requirements.
Design and implement data tools for analytics and data scientist team members to help them in building, optimizing and tuning our product.
Integrate new data management technologies and software engineering tools into existing structures.
Help in building high-performance algorithms, prototypes, predictive models and proof of concepts.
Use a variety of languages, tools and frameworks to marry data and systems together.
Recommend ways to improve data reliability, efficiency and quality.
Collaborate with Data Scientists, DevOps and Project Managers on meeting project goals.
Tackle challenges and solve complex problems on a daily basis.

The team

Advertising, Marketing & Commerce

Our Advertising, Marketing & Commerce team focuses on delivering marketing and growth objectives aligned with our clients’ brand values for measurable business growth. We do this by creating content, communications, and experiences that engage and inspire their customers to act. We implement and operate the technology platforms that enable personalized content, commerce and marketing user-centric experiences. In doing so, we transform our clients’ marketing and engagement operations into modern, data-driven, creatively focused organizations. Our team

brings deep experience in creative and digital marketing capabilities, many from our Digital Studios.

We serve our clients through the following types of work:

Cross-channel customer engagement strategy, design and development

(web, mobile, social, physical)

eCommerce strategy, implementation and operations

Marketing Content and digital asset management solutions

Marketing Technology and Advertising Technology solutions

Marketing analytics implementation and operations

Advertising campaign ideation, development and execution

Acquisition and engagement campaign ideation, development and execution

Agile based, design-thinking, user-centric, empirical projects that accelerate results

Qualifications

Required:

8+ years of experience in software development, a substantial part of which was gained in a high-throughput, decision-automation related environment.
4+ years of experience in working with big data using technologies like Spark, Kafka, Flink, Hadoop, and NoSQL datastores.
3+ years of experience on distributed, high-throughput and low-latency architecture.
1+ years of experience deploying or managing data pipelines for supporting data-science-driven decisioning at scale.
A successful track-record of manipulating, processing and extracting value from large disconnected datasets.
Producing high-quality code in Python.
Passionate about testing, and with extensive experience in Agile teams using SCRUM; you consider automated build-and-test to be the norm.
Proven ability to communicate in both verbal and writing in a high performance, collaborative environment.
Follows data development best practices, and enjoy helping others learn to do the same.
An independent thinker who considers the operating context of what he/she is developing.
Believes that the best data pipelines run unattended for weeks and months on end.
Familiar with version control, you believe that code reviews help to catch bugs, improves code base and spread knowledge.
Ability to travel 5-10% of the time

Helpful, but not required:

Knowledge in:Experience with large consumer data sets used in performance marketing is a major advantage.Familiarity with machine learning libraries is a plus.Well-versed in (or contributes to) data-centric open source projects.Reads Hacker News, blogs, or stays on top of emerging tools in some other wayData visualizationIndustry-specific marketing data

Technologies of Interest:Languages/Libraries – Python, Java, Scala, Spark, Kafka, Hadoop, HDFS, Parquet.Cloud – AWS, Azure, Google

Sr. Data Engineer

Disney Streaming Services

Disney Streaming Services is a place for the creative and the bold. Whether New York City, San Francisco, Manchester or Amsterdam, we provide opportunities to elevate your career and transform the industry.Software Engineers at Disney Streaming Services develop premium digital media products for Major League Baseball and our partners. The products we build, such as ESPN+, MLB.TV and NHL.TV are paving the way for the next-generation media and sport technologies, including the upcoming Disney+ offering. Our Engineering team for Disney Streaming Services is headquartered in the Chelsea area of New York City. Other office locations also include the SoMo area of San Francisco, CA and several international locations.

At Disney Streaming Services, data is central to measuring all aspects of the business, and critical to its operations and growth. The data engineering team is responsible for collecting, analyzing and distributing data using public cloud and open source technologies and offers transparency into customer behavior and business performance.

If you are interested in joining Disney Streaming Services in the pursuit of not only crafting new media products but enjoying the products you build, we are interested in hearing from you.

Responsibilities:

Collaborate with product teams, data analysts and data scientists to design and build data-forward solutions
Design and build and deploy streaming and batch data pipelines capable of processing and storing petabytes of data quickly and reliably
Integrate with a variety of data metric providers ranging from advertising, web analytics, and consumer devices
Build and maintain dimensional data warehouses in support of business intelligence tools
Develop data catalogs and data validations to ensure clarity and correctness of key business metrics
Drive and maintain a culture of quality, innovation and experimentation
Coach data engineers best practices and technical concepts of building large scale data platforms

Basic Qualifications:

3-5 years of experience developing in object oriented Python
Experience deploying and running AWS-based data solutions and familiar with tools such as Cloud Formation, IAM, Athena, and Kinesis
Experience engineering big-data solutions using technologies like EMR, S3, Spark and an in-depth understanding of data partitioning and sharding techniques
Familiar with metadata management, data lineage, and principles of data governance
Experience loading and querying cloud-hosted databases such as Redshift and Snowflake
Building streaming data pipelines using Kafka, Spark, or Flink

Preferred Qualifications:

Familiarity with binary data serialization formats such as Parquet, Avro, and Thrift
Experience deploying data notebook and analytic environments such as Jupyter and Databricks
Knowledge of the Python data ecosystem using pandas and numpy
Experience building and deploying ML pipelines: training models, feature development, regression testing
Experience with graph-based data workflows using Apache Airflow

Required Education:
Bachelor’s degree in Computer Science or related field or equivalent work experience

Junior Data Engineer (Contract)

ASCAP

ASCAP is home to more than 700,000 music creator members across all genres – the greatest names in music, and thousands more in the early stages of their careers. We are the world leader in performance royalties, advocacy, and service for music creators, and are the only PRO in the U.S. run by its members including songwriters, composers, and music publishers.ASCAP technologists live our mission and we are passionate about what we do for our customers and we practice what we preach. Our technologists serve with humility and a deep respect for their responsibility in helping our business partners and members achieve their goals and realize their dreams. We have an infectious and lively culture and we recognize our successes monthly at our Thursday on-site social hour celebrations. We stand behind our mission and are committed to delivering the impossible.Bottom line? We outthink ordinary. Discover what you can do with technology at ASCAP!

We are looking for someone who is passionate about data and who enjoys solving challenging problems by coming to understand and utilize complex datasets. You will be working closely with the data strategy team on high-impact revenue-generating and cost-saving projects for the business, specifically ensuring stakeholders have the data they need for analysis and can trust the integrity of the data. You will be responsible for setting up and maintaining critical data pipelines into our big data environment along with marrying various complex datasets to support analyses.

You will also be a key contributor in moving our on-premise big data environment to the cloud.

Responsibilities:

Build critical data ingestion pipelines for new datasets required to support organization-wide analytics’ needs, while ensuring data integrity is well-maintained
Support day-to-day activities of the data strategy team responsible for key revenue-driving or cost-saving analyses, specifically mining for critical datasets and/or taking the lead on complex analyses
Help drive self-service analytics throughout the organization by capturing and organizing important data and/or domain knowledge in a centralized location
Support our cloud-migration effort, moving our on-premise big data environment to the cloud

Qualifications:

You have a bachelor’s degree in computer science or equivalent experience
1-2 years of hands-on experience using SQL
You are by nature a curious individual and a lifelong learner
You are meticulous and cautious in how you approach a challenge, ensuring that you check and document your work along the way
You find it fun thinking critically and creatively with others
You have a passion for data and find yourself using it to defend your position or to support a new decision
You love music and enjoy the idea of supporting music creators

Skills:

Strong SQL knowledge
Experience in Python and/or a similar coding language
Excellent verbal and written communication skills
Exposure to Tableau or similar reporting/visualization tools

What We Love About You:

Curious: You are a lifelong learner and are driven to answer unanswered questions
Hands-On: You are willing to get your hands dirty in order to accomplish the task at hand. You have a personal mantra of “there’s always a way…”
Data-Driven: You leverage data to support your opinions and find that more detail is always preferable.
Honest: You’re willing to admit you don’t know the answer or that you have made a mistake. You consider analysis a moral endeavor and strive to prepare truthful output independent of any potential consequence.
Audible: You speak up when you disagree or don’t understand something. There are no rockstars. We want to hear what you think.
Master of Your Craft: You take pride in your work and strive to learn more to hone your skills.

Senior Data Engineer – Hux

Deloitte

What is Hux? Hux is the Human Experience Platform by Deloitte Digital.

Work you’ll do

Your responsibilities will include:

Design, construct, install, test and maintain highly scalable data pipelines with state-of-the-art monitoring and logging practices.
Bring together large, complex and sparse data sets to meet functional and non-functional business requirements.
Design and implement data tools for analytics and data scientist team members to help them in building, optimizing and tuning our product.
Integrate new data management technologies and software engineering tools into existing structures.
Help in building high-performance algorithms, prototypes, predictive models and proof of concepts.
Use a variety of languages, tools and frameworks to marry data and systems together.
Recommend ways to improve data reliability, efficiency and quality.
Collaborate with Data Scientists, DevOps and Project Managers on meeting project goals.
Tackle challenges and solve complex problems on a daily basis.

The team

Advertising, Marketing & Commerce

brings deep experience in creative and digital marketing capabilities, many from our Digital Studios.

We serve our clients through the following types of work:

Cross-channel customer engagement strategy, design and development

(web, mobile, social, physical)

eCommerce strategy, implementation and operations

Marketing Content and digital asset management solutions

Marketing Technology and Advertising Technology solutions

Marketing analytics implementation and operations

Advertising campaign ideation, development and execution

Acquisition and engagement campaign ideation, development and execution

Agile based, design-thinking, user-centric, empirical projects that accelerate results

Qualifications

Required:

8+ years of experience in software development, a substantial part of which was gained in a high-throughput, decision-automation related environment.
4+ years of experience in working with big data using technologies like Spark, Kafka, Flink, Hadoop, and NoSQL datastores.
3+ years of experience on distributed, high-throughput and low-latency architecture.
1+ years of experience deploying or managing data pipelines for supporting data-science-driven decisioning at scale.
A successful track-record of manipulating, processing and extracting value from large disconnected datasets.
Producing high-quality code in Python.
Passionate about testing, and with extensive experience in Agile teams using SCRUM; you consider automated build-and-test to be the norm.
Proven ability to communicate in both verbal and writing in a high performance, collaborative environment.
Follows data development best practices, and enjoy helping others learn to do the same.
An independent thinker who considers the operating context of what he/she is developing.
Believes that the best data pipelines run unattended for weeks and months on end.
Familiar with version control, you believe that code reviews help to catch bugs, improves code base and spread knowledge.
Ability to travel 5-10% of the time

Helpful, but not required:

Knowledge in:Experience with large consumer data sets used in performance marketing is a major advantage.Familiarity with machine learning libraries is a plus.Well-versed in (or contributes to) data-centric open source projects.Reads Hacker News, blogs, or stays on top of emerging tools in some other wayData visualizationIndustry-specific marketing data

Technologies of Interest:Languages/Libraries – Python, Java, Scala, Spark, Kafka, Hadoop, HDFS, Parquet.Cloud – AWS, Azure, Google

Sr. Data Engineer

Disney Streaming Services

If you are interested in joining Disney Streaming Services in the pursuit of not only crafting new media products but enjoying the products you build, we are interested in hearing from you.

Responsibilities:

Collaborate with product teams, data analysts and data scientists to design and build data-forward solutions
Design and build and deploy streaming and batch data pipelines capable of processing and storing petabytes of data quickly and reliably
Integrate with a variety of data metric providers ranging from advertising, web analytics, and consumer devices
Build and maintain dimensional data warehouses in support of business intelligence tools
Develop data catalogs and data validations to ensure clarity and correctness of key business metrics
Drive and maintain a culture of quality, innovation and experimentation
Coach data engineers best practices and technical concepts of building large scale data platforms

Basic Qualifications:

3-5 years of experience developing in object oriented Python
Experience deploying and running AWS-based data solutions and familiar with tools such as Cloud Formation, IAM, Athena, and Kinesis
Experience engineering big-data solutions using technologies like EMR, S3, Spark and an in-depth understanding of data partitioning and sharding techniques
Familiar with metadata management, data lineage, and principles of data governance
Experience loading and querying cloud-hosted databases such as Redshift and Snowflake
Building streaming data pipelines using Kafka, Spark, or Flink

Preferred Qualifications:

Familiarity with binary data serialization formats such as Parquet, Avro, and Thrift
Experience deploying data notebook and analytic environments such as Jupyter and Databricks
Knowledge of the Python data ecosystem using pandas and numpy
Experience building and deploying ML pipelines: training models, feature development, regression testing
Experience with graph-based data workflows using Apache Airflow

Required Education:
Bachelor’s degree in Computer Science or related field or equivalent work experience

Junior Data Engineer (Contract)

ASCAP

You will also be a key contributor in moving our on-premise big data environment to the cloud.

Responsibilities:

Build critical data ingestion pipelines for new datasets required to support organization-wide analytics’ needs, while ensuring data integrity is well-maintained
Support day-to-day activities of the data strategy team responsible for key revenue-driving or cost-saving analyses, specifically mining for critical datasets and/or taking the lead on complex analyses
Help drive self-service analytics throughout the organization by capturing and organizing important data and/or domain knowledge in a centralized location
Support our cloud-migration effort, moving our on-premise big data environment to the cloud

Qualifications:

You have a bachelor’s degree in computer science or equivalent experience
1-2 years of hands-on experience using SQL
You are by nature a curious individual and a lifelong learner
You are meticulous and cautious in how you approach a challenge, ensuring that you check and document your work along the way
You find it fun thinking critically and creatively with others
You have a passion for data and find yourself using it to defend your position or to support a new decision
You love music and enjoy the idea of supporting music creators

Skills:

Strong SQL knowledge
Experience in Python and/or a similar coding language
Excellent verbal and written communication skills
Exposure to Tableau or similar reporting/visualization tools

What We Love About You:

Curious: You are a lifelong learner and are driven to answer unanswered questions
Hands-On: You are willing to get your hands dirty in order to accomplish the task at hand. You have a personal mantra of “there’s always a way…”
Data-Driven: You leverage data to support your opinions and find that more detail is always preferable.
Honest: You’re willing to admit you don’t know the answer or that you have made a mistake. You consider analysis a moral endeavor and strive to prepare truthful output independent of any potential consequence.
Audible: You speak up when you disagree or don’t understand something. There are no rockstars. We want to hear what you think.
Master of Your Craft: You take pride in your work and strive to learn more to hone your skills.