Senior ML Infrastructure Engineer Job at Hippocratic AI, Palo Alto, CA

NG42aFdOaDIwYUlkdlNLRXdqT21xdmJPc0E9PQ==
  • Hippocratic AI
  • Palo Alto, CA

Job Description

About Us:

Hippocratic AI has developed a safety-focused Large Language Model (LLM) for healthcare. The company believes that a safe LLM can dramatically improve healthcare accessibility and health outcomes in the world by bringing deep healthcare expertise to every human. No other technology has the potential to have this level of global impact on health. 

Why Join Our Team:

  • Innovative Mission: We are developing a safe, healthcare-focused large language model (LLM) designed to revolutionize health outcomes on a global scale.

  • Visionary Leadership: Hippocratic AI was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from leading institutions, including El Camino Health, Johns Hopkins, Stanford, Microsoft, Google, and NVIDIA.

  • Strategic Investors: We have raised a total of $278 million in funding, backed by top investors such as Andreessen Horowitz, General Catalyst, Kleiner Perkins, NVIDIA’s NVentures, Premji Invest, SV Angel, and six health systems.

  • World-Class Team: Our team is composed of leading experts in healthcare and artificial intelligence, ensuring our technology is safe, effective, and capable of delivering meaningful improvements to healthcare delivery and outcomes.

Position Overview:

We are seeking a skilled ML Infrastructure Engineer to help design, build, and maintain a robust orchestration platform for managing a diverse set of Large Language Models (LLMs). The ideal candidate will have hands-on experience with infrastructure orchestration tools such as Kubernetes and Terraform, as well as a strong understanding of multi-cloud environments. This role offers the opportunity to work on cutting-edge technologies and play a key part in scaling our AI infrastructure.

Key Responsibilities: Infrastructure Development & Maintenance:

• Build and maintain infrastructure for deploying and managing LLMs at scale.
• Implement automated processes using Kubernetes and Infrastructure as Code (IAC) tools like Terraform.

Orchestration Platform Support:

• Contribute to the development and optimization of an orchestration platform for managing a heterogeneous set of LLMs.
• Monitor and troubleshoot issues in the platform to ensure high availability and performance.

Cloud Integration:

• Deploy and manage resources across multiple cloud platforms (e.g., AWS, Azure, Google Cloud).
• Optimize cloud resource usage for cost efficiency and scalability.

Collaboration:

• Work closely with ML engineers and DevOps teams to ensure smooth deployment and operation of AI models.
• Provide feedback on system designs and recommend improvements to infrastructure workflows.

Performance Monitoring:

• Implement tools and processes to monitor system health, identify bottlenecks, and improve model lifecycle management.
• Perform capacity planning to support growing infrastructure needs.

Qualifications:

Technical Skills:

• 3-5 years of experience in infrastructure engineering, DevOps, or a related field.

  • Experience with enterprise GPUs such as H200, H100, A100

• Proficiency with Kubernetes, Terraform, and other IAC tools.
• Familiarity with multi-cloud environments and cloud-native services (e.g., AWS Lambda, Google Cloud Run, Azure Functions).
• Programming skills in Python, Bash, or a similar language for automation and scripting.
• Basic understanding of ML workflows and frameworks like TensorFlow, PyTorch, or Hugging Face is a plus.

Soft Skills: • Strong problem-solving skills and attention to detail.
• Good communication and collaboration abilities to work effectively with cross-functional teams.
• Eagerness to learn new technologies and improve existing systems.

Education & Experience: • Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent work experience).

Job Tags

Remote job, Work experience placement,

Similar Jobs

DoubleTree by Hilton Hotel Denver

Full Time Shuttle Driver - CDL Class B Required Job at DoubleTree by Hilton Hotel Denver

 ...for a highly motivated individuals with a valid Commercial Drivers License (CDL-B) to join our Guest Services team as a hotel Shuttle Driver....  ...driven person to grow a career in hospitality with a world-class property. Your Role: Help guests discover their Wanderlust... 

Montefiore Health System

PHYSICIAN ASSISTANT PER DIEM (JR226911) Job at Montefiore Health System

 ...offer our associates, please click here. The practice of a physician assistant shall provide medical services within the education,...  ...to the Core Skills listed here, PAs will request validation to perform department specific skills. This position is per-diem.

Tap Growth ai

Box Truck Driver Job at Tap Growth ai

 ...We're Hiring: Box Truck Driver! We are seeking a reliable and experienced Box Truck Driver to join our transportation team. The ideal candidate will have a strong safety record, excellent driving skills, and the ability to handle delivery operations efficiently while... 

Rankings Io

Financial Analyst Job at Rankings Io

 ...We are seeking a motivated and detail-oriented Financial Analyst to join our finance team at Rankings.io. Reporting directly to the...  ...Starting from $80k; pay commensurate with experience ~ Work remotely from home ~ Unlimited PTO ~1 Wellness Day per Month (in... 

Wyndy

Evening Babysitter for School-age Children Job at Wyndy

We are seeking a responsible and engaging babysitter for an evening job in New Orleans, LA. The babysitting session will take place on Saturday, October 11, 2025, starting at 10:00 PM and ending on Sunday, October 12, 2025, at 2:00 AM. You will be caring for three children...