Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises 100+ professionals across Machine Learning, Data Engineering & Cloud expertise.
We are looking to hire big data lead who will be working with our global clientele to help architect & develop data ingestion, transformation, and storage solutions using open sources and cloud technologies in the supply chain domain. Below are important responsibilities and skills for this role
Experience & Skills
Overall, 10 years plus experience with a minimum of 3 years designing and architecting data pipelines for ingestion and transformation of structured & unstructured data sources using Hadoop, Hive, Pyspark, and Airflow tools.
Deep understanding of the ETL project life cycle; including data modeling, data quality monitoring, error handling, failure mitigation, project management & documentation.
At least 5-6 big data projects in a lead role with global customers is preferred
Understanding of various cloud platforms including Azure, AWS & GCP for constructing data pipelines and storage using cloud-native services. You should be strong in 1 out of three cloud platforms.
Understanding of database and analytical technologies in the industry including NoSQL databases, Data Warehouse, BI reporting, and Dashboard development.
Customer-facing skills to represent Valiance well within the customer’s environment and drive discussions with senior personnel regarding trade-offs, best practices, project management, and risk mitigation.
Should be able to propose and recommend technology architecture, choice of technology platforms based on different workloads, cost constraints and client requirements.
Coach and mentor engineers to raise the technical ability of the rest of the team, and/or to become certified in required AWS & GCP technical certifications.
Become a thought leader in data engineering for Valiance; representing the company in different technology events, support in pre-sales activities, build and grow data engineering practice.
Technical Skills: Python, SQL, Airflow, NIFI, PySpark, Hadoop, Hive, HDFS, Cloud Platforms: GCP, AWS, Azure
Certifications: Cloud certifications are preferred