Lead Big Data Engineer

San Francisco, CA, USA

Ranked as #12 on Forbes’ List of 25 Fastest Growing Public Tech Companies for 2017, EPAM is committed to providing our global team of 25,900+ EPAMers with inspiring careers from day one. EPAMers lead with passion and honesty and think creatively. Our people are the source of our success and we value collaboration, try to always understand our customers’ business, and strive for the highest standards of excellence. No matter where you are located, you’ll join a dedicated, diverse community that will help you discover your fullest potential.


You are curious, persistent, logical and clever – a true techie at heart. You enjoy living by the code of your craft and developing elegant solutions for complex problems. If this sounds like you, this could be the perfect opportunity to join EPAM as a Lead Big Data Engineer. Scroll down to learn more about the position’s responsibilities and requirements.

As a Lead Big Data Engineer, you will lead, design and implement innovative analytical solutions and work alongside the product engineering team, evaluating new features and architecture. Reporting to the Director, Analytics and Personalization, you will help drive decisions and build collaborative relationships with key functional groups in the organization.

What You’ll Do

  • Lead, design and implement innovative analytical solution using Hadoop, NoSQL, and other Big Data related technologies;
  • Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions;
  • Build collaborative partnerships with architects, technical leads and key individuals within other functional groups;
  • Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution;
  • Actively participate in code review and test solutions to ensure it meets best practice specifications;
  • Build and foster a high performance engineering culture, mentor team members and provide team with the tools and motivation to make things happen;
  • Work with stakeholders and cross-functional teams to develop new solutions or enhance existing solution.

What You Have

  • 5+ years of software development and deployment experience with at least 2 years of hands-on experience with Hadoop applications (e.g. administration, configuration management, monitoring, debugging, and performance tuning);
  • Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), data warehouse or database architecture;
  • Solid, in depth understanding of Hadoop, Spark and Hive;
  • Strong experience with data modeling;
  • Hands-on development experience using open source big data components such as Hadoop, Hive, Pig, Spark, HBase, Hawk, Oozie, Mahout, Flume, Kafka, ZooKeeper, Sqoop etc. preferably with Hortonworks distro;
  • Microsoft Azure: Experience designing, deploying, and administering scalable, available, and fault tolerant systems on Microsoft Azure using HDInsights or Analytics Platform System (APS);
  • Experience with Azure Management Portal, Azure Machine Learning, and Azure SQL Server;
  • Solid understanding of DevOps principles and practices;
  • Experience with Scala/Java/Python (any combination);
  • Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills;
  • Data-oriented personality. Motivated, independent, efficient and able to handle several projects; work under pressure with a solid sense for setting priorities;
  • Ability to work in a fast-paced (startup like) agile development environment;
  • Friendly, articulate, and interested in working in a fun, small team environment;
  • Experience working in the retail industry with a large-scale enterprise organization, ecommerce, marketing and CRM applications will be a plus;
  • BS or MS in Computer Science or equivalent; MS preferred.

What We Offer

  • Medical, Dental and Vision Insurance (Subsidized);
  • Health Savings Account;
  • Flexible Spending Accounts (Healthcare, Dependent Care, Commuter);
  • Short-Term and Long-Term Disability (Company Provided);
  • Life and AD&D Insurance (Company Provided);
  • Matched 401(k) Retirement Savings Plan;
  • Paid Time Off;
  • Legal Plan and Identity Theft Protection;
  • Accident Insurance;
  • Employee Discounts;
  • Pet Insurance.