Azure AI / ML Engineer (m/f/d) – Large Language Models, Conversational AI, Azure - 60% Remote 40% in Vienna

Azure AI / ML Engineer (m/f/d) – Large Language Models, Conversational AI, Azure - 60% Remote 40% in Vienna 
Vienna (Hybrid) | Long-term Program (3–5 years) | English or German

For a long-term IT and AI transformation program, we are seeking an experienced Azure AI / ML Engineer (m/f/d) to design, develop, implement, and operate enterprise-grade AI solutions on Microsoft Azure.
The role focuses strongly on Large Language Models (LLMs), Conversational AI, Retrieval-Augmented Generation (RAG) applications, AI agents and multi-agent systems, and the productive operation of scalable, secure Azure architectures.

As an Azure AI / ML Engineer, you will work closely with business units and IT teams, supporting the full lifecycle of AI solutions — from requirements analysis and solution design through to production rollout, operational stability, and knowledge transfer.This is a 100% workload, long-term engagement, offering a hybrid working model in Vienna with significant remote flexibility.

Project Details

• Start Date: March / April 2026
• Duration: 3 years (with extension option up to 5 years)
• Workload: Full-time (100%)
• Location: Vienna, Austria (approx. 40% on-site, 60% remote)
• Project Language: English or German

Key Responsibilities

• Design, develop, and implement LLM-based Conversational AI solutions (e.g. RAG applications) on Microsoft Azure
• Build, orchestrate, and continuously improve AI agents and multi-agent systems
• Apply and integrate Model Context Protocols (MCPs)
• Gather, analyse, and structure business and technical requirements in close collaboration with stakeholders
• Deliver AI solutions using agile methodologies and continuously enhance them in coordination with business teams
• Integrate and harmonise internal and external data sources
• Support rollout, deployment, and operation of AI solutions in production environments
• Transfer solutions to internal teams, including documentation, upskilling, and user enablement
• Ensure scalability, security, maintainability, and operational stability of Azure AI solutions

Required Skills & Experience

• Strong hands-on experience with Large Language Models (LLMs) and Conversational AI on Microsoft Azure
• Practical experience with RAG (Retrieval-Augmented Generation) architectures
• Proven experience designing and implementing AI agents and multi-agent systems
• Experience working with Model Context Protocols (MCPs)
• Strong background in requirements engineering and collaboration with business stakeholders
• Experience delivering solutions in agile environments
• Solid experience integrating and harmonising data sources
• Experience operating and supporting production-grade AI applications
• Experience with solution handover, documentation, and knowledge transfer

Technical Requirements (Must-Have)

• Very strong expertise in Microsoft Azure, especially:
  
  • Microsoft Foundry
  • Azure Cognitive Services
  • Azure Machine Learning
• Strong development skills in C# / .NET and Python
• Experience with DevOps, CI/CD pipelines, and automation
• Hands-on experience with Docker and Azure Kubernetes Service (AKS)
• Experience with Infrastructure as Code (IaC)
• High learning agility and strong interest in Artificial Intelligence, Machine Learning, and Generative AI

Nice to Have (Plus Skills)

• Experience with Confluence and JIRA
• Knowledge of Azure API Management, Azure Monitor, Log Analytics, and Microsoft Purview (Data Governance)
• Industry experience in the energy sector or public sector
• Structured, self-driven, and solution-oriented working style
• Strong communication skills with both technical and non-technical stakeholders
• Team-oriented mindset and strong focus on quality and reliability

Why Apply?

• Work on a strategic, long-term Azure AI program
• Hands-on involvement with cutting-edge Generative AI, LLMs, and Conversational AI
• Hybrid working model with significant remote flexibility
• Opportunity to shape enterprise-scale AI architectures in a production environment
• Stable, multi-year engagement in an international project setup