Sr. Software Engineer, Data Lake

Senior Software Engineer, Data Lake

 (Job Location: Emeryville, CA) – 25220


Founded in 1923, Nielsen is the global authority on the consumer.  We help over 20,000 clients, from Coca-Cola to Walmart to Apple to Disney, in 105 countries understand consumers and grow their businesses, every day.  The company is organized into two areas: What Consumers Watch, serving media, telecom and tech industries; and What Consumers Buy, serving consumer goods, retail, financial services and automotive industries. 


In this role, you will be responsible for:


Responsible for building a big data analytics platform at scale on the public cloud, and deriving value from the data using data science methods, for web consumer and open source companies: Gather and analyze client requirements from a designer’s perspective, and create a configurable solution, to meet project goals. Develop, create, test, modify and support complex dashboard and software applications for market research, using Agile methodology. Build data pipelines to integrate data from multiple sources into a single platform, making it accessible by the end user. Lead end-to-end integration efforts and set-up environments of highly complex applications. Lead troubleshooting of server and data performance issues. Write unit tests, and lead code reviews, taking necessary corrective actions. Work closely with the software development team to put together seemingly disparate pieces of data, build out the data warehouse and create interactive dashboards, and to implement reusable system solutions. Provide project updates to the team lead and product manager. Diagnose production issues and provide solutions. Evaluate new technologies, products and frameworks, and work closely with product managers and architects to ensure successful adoption of the same in the final product. Develop standardized dashboards and ad-hoc reports as a basis for business decisions, with an emphasis on automation and scalability. Tools and technologies used: Hadoop, Kafka, Amazon Web Services (AWS), Elasticsearch, Python, Spark SQL, JavaScript, Angular JS, S3, Ec2, NiFi, Logstash, Flume, Kibana, Tableau, Oracle, and SQL-Server2012.





Minimum Requirements:

  • Master’s degree in computer science, information technology management, engineering, or a related field (foreign equivalent degree acceptable), plus 3 years of experience in Java software development (would also accept a bachelor’s degree plus 5 years of progressively responsible post-baccalaureate experience).   
  • This must include:
    • 3 years of experience in:
      • big data stack data flows and architecture;
      • ETL (extract-transform-load); processing of structured and unstructured data;
      • object-oriented programming and design;
      • statistics-based data mining concepts and visualization tools; and
      • Hadoop.
    • Must have 2 years of experience in:
      • Elasticsearch;
      • Python;
      • S3; and
      • Ec2.
    • Must have 1 year of experience in:
      • Spark SQL;
    • Any level of experience in and/or knowledge of:
      • Kafka;
      • Javascript;
      • Angular JS;
      • NiFi;
      • Logstash;
      • Flume;
      • Kibana;
      • Tableau;
      • SQL-Server2012 and/or other ingestion technologies;
      • working with teams located in geographically diverse areas; and
      • applying analytical and problem-solving skills to solve complex design and coding problems.


Nielsen is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class.


JOB TYPE:  Regular

PRIMARY LOCATION:  North America-US-Emeryville, CA

ORGANIZATION:  Technology and Engineering

SCHEDULE:  Full-time



Full time
Emeryville, CA 94608, US