• Software Development Engineer – Big Data, AWS Elastic MapReduce (EMR)

    Location US-CA-East Palo Alto
    Posted Date 2 weeks ago(1/4/2019 2:44 PM)
    Job ID
    Company/Location (search) : Country (Full Name)
    United States
  • Job Description

    Want to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!

    Amazon EMR is a web service which enables customers to run massive clusters with distributed big data frameworks like Apache Hadoop, Hive, Tez, Flink, Spark, Presto, HBase and more, with the ability to effortlessly scale up and down as needed. We run large number of customer clusters, enabling processing on vast datasets.

    We are developing innovative new features including our next-generation cluster management system, improvements for real-time processing of big data, and ways to enable customers to more easily interact with their data. We’re looking for top engineers to build them from the ground up.

    Here are sample features that we have recently delivered:
    • Authenticate with Kerberos: https://aws.amazon.com/about-aws/whats-new/2017/11/now-enable-kerberos-authentication-and-emrfs-authorization-in-amazon-emr/
    • EMR Per Second Billing: https://aws.amazon.com/about-aws/whats-new/2017/10/amazon-emr-now-supports-per-second-billing/
    • Spark Application History in EMR Console: https://aws.amazon.com/about-aws/whats-new/2017/09/now-view-apache-spark-application-history-and-yarn-application-status-in-the-amazon-emr-console/
    • Custom AMIs: https://aws.amazon.com/blogs/big-data/create-custom-amis-and-push-updates-to-a-running-amazon-emr-cluster-using-amazon-ec2-systems-manager/
    • EMR Instance Fleets: https://aws.amazon.com/blogs/aws/new-amazon-emr-instance-fleets/
    • Auto Scaling EMR Clusters: https://aws.amazon.com/blogs/big-data/dynamically-scale-applications-on-amazon-emr-with-auto-scaling/
    • EMR Security Configurations: https://aws.amazon.com/blogs/big-data/encrypt-data-at-rest-and-in-flight-on-amazon-emr-with-security-configurations/
    • Responding To State Changes with AWS CloudWatch Events: https://aws.amazon.com/blogs/big-data/respond-to-state-changes-on-amazon-emr-clusters-with-amazon-cloudwatch-events/
    This is a hands-on position where you will be do everything from designing and building extremely stable components and cutting-edge features for the savviest customers in the business to help them get the best results.

    You will have a chance to work with the open source community and contribute significant portions its software to open source projects possibly including Apache Hadoop, Spark, Pig and Hbase. You need to not only be a top software developer with excellent programming skills, an understanding of big data and parallelization, and a stellar record of delivery but also excel at leadership and customer obsession and have a real passion for massive-scale computing. If you want to truly test your mettle against the hardest challenges in distributed systems to build solutions for large scale problems in a wide variety of domains, come join our group.

    Your responsibilities will include:
    - Translation of complex functional and technical requirements into detailed architecture and design
    - Deliver systems and features with top-notch quality, on time
    - Develop new technologies for monitoring production clusters
    - Own the software development process end-to-end, including: working with engineers and product managers to develop requirements; designing, architecting, planning, implementing, and testing new systems and features; deploying, and operating the production EMR systems.

    In joining our team, you will get to work with a minimum of technical supervision, while playing a variety of roles as needed to respond efficiently to multiple program priorities. You will get to collaborate with some of the best and brightest minds in the industry. You'll enjoy a competitive salary, great benefits, a creative and agile work environment, and the exciting opportunity to be part of a fast-paced and growing team and one of the most innovative technology companies - but most of all, you will get the satisfaction of making products that millions use everyday to great effect!

    For more information:
    • AWS re:Invent 2017 Amazon EMR: https://www.youtube.com/watch?v=1CAWf9VDgFM
    • AWS re:Invent 2017 Keynote - Andy Jassy: https://www.youtube.com/watch?v=1IxDLeFQKPk
    • AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
    • AWS EMR: https://aws.amazon.com/emr/

    Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation

    Basic Qualifications

    • Master's degree in Computer Science or equivalent with 2+ years of experience OR Bachelor's Degree in Computer Science or equivalent with 4+ years experience, in: software development, including design, implementation, debugging, and support
    • Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
    • Strong proficiency in developing objected-oriented software, with deep experience in one or more relevant languages (Java, C, C++, C#)

    Preferred Qualifications

    • Very strong Computer Science fundamentals in algorithm design, data structures, problem solving, and complexity analysis
    • Experience designing and building highly-scaled distributed systems and web services
    • Thorough understanding of parallel algorithms, concurrency, asynchronous architectures
    • Experience working in an agile software development organization
    • Experience building with SOA using Java on Linux
    • Proficiency in high-performance, multi-threaded programming
    • Knowledge of and contribution to Hadoop ecosystem
    • Experience with distributed systems architecture
    • Experience with one or more of Ruby, Python, Perl
    • Proven ability to effectively drive cross-team solutions that may have complex dependencies
    • Masters in Computer Science with emphasis on distributed systems and Big Data architectures is a plus.
    Sorry the Share function is not working properly at this moment. Please refresh the page and try again later.
    Share this job