Description

As a Senior Data Engineer within our Delivery Intelligence Group, you will be responsible for architecting optimal data workflows and associated cloud infrastructure that support BrightDrop’s core analytic products and services. You will work closely with our data scientists, DevOps, and software engineers to automate pipelines that rest data in the BrightDrop Data Lake and enable exploration, visualization, and analysis tools and workflows for machine learning, optimization, and AI applications. The ideal candidate will have experience standing up ETL pipelines to handle massive volumes of data, implementing and maintaining workflow orchestration systems using Spark-based data platforms, and working with time-series and geospatial databases.

Responsibilities:

Architect, implement, and maintain robust data acquisition and ETL pipelines in Azure.
Lead data strategy and governance for delivery intelligence and modeling products
Use Python and Scala-based tools as well as open source relational and NoSQL databases to access, manipulate, and join external datasets to internal data (e.g., via REST APIs, PySpark)
Assist our data scientists in production deployments of machine learning models and analytics products (e.g., dashboards).
Generate and deploy feature engineering pipelines to augment data for use in downstream analysis tasks.
Work with information security to define and implement data classification, logging, and access control.

Additional Description

Required:

Bachelors degree (Master’s preferred) or equivalent experience in computer science, data science, engineering, or related quantitative field
5+ years of industry experience developing, implementing, and maintaining solutions for Big Data or data warehousing systems
3+ years of industry experience working in the Azure cloud environment
2+ years of experience working with Scala and Python-based event-based and/or streaming analytics solutions (e.g., Spark-streaming or Kafka)
Basic understanding of machine learning/statistical learning principles
Advanced experience working with containerization technologies and orchestration platforms (specifically Docker and Kubernetes).
Experience implementing and maintaining data workflow tools at scale (e.g. Databricks, Airflow, Prefect, Luigi, AWS step functions, etc.)
Experience working in teams using agile software development methodologies together with distributed version control systems (e.g., Jira + GitHub/Bitbucket)
Understanding of and experience with application of automated testing and CI/CD in functional deployment environments.
Self-driven with an interest in on-the-job learning.

Preferred:

Experience integrating simulation systems with distributed, data-intensive processing or analytics applications
Advanced experience working with relational databases and SQL query authoring

Desired:

Working familiarity with a variety of other database technologies (e.g., geospatial, time-series, and graph databases).
Domain knowledge in transportation and energy systems, graph algorithms, convex optimization, and/or reinforcement learning
Passion for transportation decarbonization
Adherence to clean code principles

#LI - Hybrid

The expected base compensation for this role is: (90,223 - 144,123 USD Annual). Actual base compensation within the identified range will vary based on factors relevant to the position.

Bonus Potential: An incentive pay program offers payouts based on company performance, job level, and individual performance.

Benefits: GM offers a variety of health and wellbeing benefit programs. Benefit options include medical, dental, vision, Health Savings Account, Flexible Spending Accounts, retirement savings plan, sickness and accident benefits, life insurance, paid vacation & holidays, tuition assistance programs, employee assistance program, GM vehicle discounts and more.

About GM

Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace the responsibility to lead the change that will make our world better, safer and more equitable for all.

Why Join Us

We aspire to be the most inclusive company in the world. We believe we all must make a choice every day – individually and collectively – to drive meaningful change through our words, our deeds and our culture. Our Work Appropriately philosophy supports our foundation of inclusion and provides employees the flexibility to work where they can have the greatest impact on achieving our goals, dependent on role needs. Every day, we want every employee, no matter their background, ethnicity, preferences, or location, to feel they belong to one General Motors team.

Total Rewards | Benefits Overview

From day one, we're looking out for your well-being–at work and at home–so you can focus on realizing your ambitions. Learn how GM supports a rewarding career that rewards you personally by visiting Total Rewards resources.

Diversity Information

General Motors is committed to being a workplace that is not only free of discrimination, but one that genuinely fosters inclusion and belonging. We strongly believe that workforce diversity creates an environment in which our employees can thrive and develop better products for our customers. We understand and embrace the variety through which people gain experiences whether through professional, personal, educational, or volunteer opportunities. GM is proud to be an equal opportunity employer.

We encourage interested candidates to review the key responsibilities and qualifications and apply for any positions that match your skills and capabilities.

Equal Employment Opportunity Statements

The policy of General Motors is to extend opportunities to qualified applicants and employees on an equal basis regardless of an individual's age, race, color, sex, religion, national origin, disability, sexual orientation, gender identity/expression or veteran status. Additionally, General Motors is committed to being an Equal Employment Opportunity (EEO) Employer and offers opportunities to all job seekers including individuals with disabilities. If you need a reasonable accommodation to assist with your job search or application for employment, email us at Careers.Accommodations@GM.com or call us at 800-865-7580. In your email, please include a description of the specific accommodation you are requesting as well as the job title and requisition number of the position for which you are applying.