Job Overview
We are looking for a Data Scientist with strong experience in Generative AI and Large Language Models (LLMs) to design, build, and deploy intelligent AI-powered applications. The ideal candidate should have hands-on expertise in LLM integration, Retrieval-Augmented Generation (RAG), prompt engineering, and AI model evaluation while ensuring safety, accuracy, and performance optimization.
The role involves working closely with data engineers, ML engineers, and product teams to develop scalable GenAI solutions deployed on cloud platforms such as Azure OpenAI.
Key Responsibilities
Generative AI & LLM Development
- Design, build, and optimize LLM-based applications using platforms like OpenAI / Azure OpenAI and locally hosted models.
- Develop and refine prompt engineering strategies and system prompts to improve response accuracy and reliability.
- Implement Retrieval-Augmented Generation (RAG) pipelines to enable context-aware responses.
RAG & Knowledge Integration
- Build scalable retrieval systems using techniques such as document chunking, embeddings, and semantic search.
- Implement and manage vector databases such as FAISS or Pinecone for knowledge retrieval.
- Integrate structured and unstructured data sources into GenAI pipelines.
Frameworks & AI Development
- Develop AI workflows using frameworks like LangChain and LlamaIndex.
- Build modular pipelines to support conversational AI, document intelligence, and knowledge assistants.
Model Evaluation & Experimentation
- Evaluate LLM outputs using quantitative metrics such as BLEU, BERTScore, and other benchmarking techniques.
- Implement human-in-the-loop evaluation frameworks to improve model quality.
- Track experiments and model performance using MLflow or similar tools.
AI Safety & Responsible AI
- Implement guardrails and safety mechanisms to handle toxicity, bias, and harmful outputs.
- Detect and manage PII (Personally Identifiable Information) in AI responses.
- Apply techniques for grounding, citation, and hallucination mitigation to ensure reliable AI outputs.
Deployment & Optimization
- Deploy scalable GenAI services on Azure, particularly using Azure OpenAI (AOAI).
- Optimize latency, cost, and inference performance for production systems.
- Monitor system performance and continuously improve reliability.
Required Skills & Technologies
Programming & AI
- Python
- NLP libraries such as spaCy, Hugging Face Transformers
- Data processing and model experimentation
GenAI & LLM Technologies
- OpenAI / Azure OpenAI APIs
- Prompt engineering and system prompt design
- Retrieval-Augmented Generation (RAG)
Frameworks
- LangChain
- LlamaIndex
Vector Databases
- FAISS
- Pinecone
Evaluation & Experiment Tracking
- BLEU, BERTScore
- MLflow or similar experiment tracking tools
Cloud & Deployment
- Azure
- Azure OpenAI services
- Model deployment and monitoring
Preferred Qualifications
- Experience building enterprise GenAI applications such as chatbots, document assistants, or knowledge search platforms.
- Understanding of MLOps and LLMOps practices.
- Experience with fine-tuning or parameter-efficient tuning techniques.
- Familiarity with data governance, AI ethics, and responsible AI frameworks.
