Returning Candidate?

Data Engineer- Internet Monitoring

Data Engineer- Internet Monitoring

Job ID 
Posted Date 

Job Description

While we tend to think of it as single entity, the Internet is actually comprised of tens of thousands of independently-administered networks, with routing protocols facilitating the constant flow of traffic within and between them. As if that weren’t complicated enough, the flow of traffic across the Internet is also subject to various political, regulatory, and business requirements, not to mention the traffic engineering done by the large operators and players. The Internet is amazingly complex and even when things are humming along smoothly, we make it our mission to study its behavior and how it breaks. More specifically, we study how Amazon’s network connects to the public Internet in order to detect when those connections are disrupted and impact our customers. In light of this, we are looking for a Data Engineer to join our R&D effort and help drive insight into this fruitful and wide-open problem space.

The ideal candidate will have a demonstrated affinity for engineering best practices, data management fundamentals, data storage principles, and be current on recent advances in distributed systems as it pertains to data storage and computing. Building and operating scalable infrastructure for data analysis, you strive to answer the “big” questions with data

Basic Qualifications

  • Responsible for designing, building, and maintaining an efficient, extensible, and scalable data infrastructure(s) for processing high-volume and high-velocity data
  • Proven track record of processing unstructured and/or complex semi-structured data streams and building self-service business intelligence infrastructure
  • Proven track record in building, operating and optimizing distributed, large-scale data storage and analysis solutions (e.g. RedShift, Dynamo, Cassandra, Spark, EMR, Hadoop)
  • Experience programming in a general-purpose language (e.g. Java, Scala, Python, Go, Ruby, C/C++, JavaScript, etc.)
  • Knowledge of relational data modeling concepts and basic SQL and data analysis skills
  • Strong problem-solving and troubleshooting abilities
  • Knowledge of data storage best practices and use cases

Preferred Qualifications

  • Familiarity with streaming data analysis solutions (e.g. Kafka, Kinesis)
  • Experience in ETL pipelining and data warehousing
  • Web development and Data Visualization experience a plus
  • Familiarity with networking protocols (TCP/IP, ICMP, BGP)