10 days old

Senior Data Engineer

Houston, TX 77002


We have a wide variety of career opportunities around the world — come find yours.



The United IT team designs, develops and maintains massively scaling technology solutions that are brought to life with innovative architectures, data analytics and digital solutions.


Job overview and responsibilities

This role will be responsible for all activities related to the design, operations, implementation, and 24x7 production support of United's complex, highly-available Kafka, Elastic and Hadoop ecosystem. Responsibilities include  infrastructure and operations support, working with development teams on design, modeling, and implementation of data pipelines utilizing Kafka messaging and Elastic search data store.

This is a hands-on role with team lead responsibilities and requires strong drive to find the optimal solutions  to evangelize Kafka and Elastic technology with United Airlines, Inc.

Kafka  and Elastic Search Administration and support.

  • Production support of Kafka and Elastic search
  • Install, configure, administer and support multiple Kafka and ELK clusters.
  • Perform maintenance and troubleshooting. Participate in capacity planning and growth projections
  • Support application deployments on production including performance and functional testing.
  • Automate routine tasks with scripts
  • Monitor and maintain system health and ensure efficient operation
  • Work with cross functional teams planning, modeling, and implementing solutions utilizing Kafka and ELK technologies.
  • Work closely with Big Data and AWS cloud technology groups.
  • Experience with open source Kafka distributions as well as enterprise Kafka products preferred.
  • Familiarity with both cloud native Kafka (on AWS) and on-premise architectures.
  • Managing Kafka & Elastic search clusters & creating tools to automate and improve clusters' reliability and performance.

Lead team on Kafka and Elastic implementations.

  • Lead and participate in On Call rotation for production support.



  • BS in Computer Science and/or equivalent work experience required
  • 3+ years of experience supporting Linux environments.
  • 3+ years of experience with Shell scripting    
  • 3+ years of experience with python and SQL
  • 3+ years of related work experience
  • Strong data analysis skills; ability to independently write scripts/code to parse and analyze complex data
  • Excellent verbal and written communication skills
  • Communicate and work effectively in a team environment
  • Must be legally authorized to work in the United States for any employer without sponsorship
  • Successful completion of interview required to meet job qualification
  • Reliable, punctual attendance is an essential function of the position


  • Experience with configuration management tools like Ansible is a plus.
  • Elastic search Engineer I or II certification.
  • Kafka Operations and Administration Certification.
  • Experience architecting and implementing Kafka clusters in a large-scale enterprise environment.
  • Experience with Elastic (Elastic Search, Logstash, Kibana) in a large-scale enterprise environment.
  • Experience with Big Data and NoSQL technologies a plus
  • Experience in parsing logs with GROK and regex
  • Experience in Java, Python, Apache, Confluent Kafka,  SQL, Hadoop
  • Experience in Configuration Management (Puppet, Chef, Ansible), Source code Management (GitHub/Gitlab)
  • Experience with other features of Elastic Stack (Canvas, Elasticsearch-SQL, and Elasticsearch-Hadoop,) 


Equal Opportunity Employer – Minorities/Women/Veterans/Disabled/LGBT


Posted: 2020-02-07 Expires: 2020-03-08

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

Senior Data Engineer

Houston, TX 77002

Join us to start saving your Favorite Jobs!

Sign In Create Account
Powered ByCareerCast