Artificial intelligence (AI) has demonstrated increasing potential to assist humans in decision-making tasks. However, AI systems solve problems differently than humans, leading to varying performance in human-AI teams. An ideal human-AI collaboration system should leverage the strengths of both humans and AI while mitigating their weaknesses. In this dissertation, I will discuss how to empower humans with AI systems in decision-making tasks. First, I will explain how to understand human-AI teams under different distribution types and interactive interfaces (Chapter 2). Then, I will describe how to align AI models with human perceptions for better decision support (Chapter 3). Third, I will explore how to build AI-driven tutorials using selected concepts and examples to assist and teach humans in fine-grained image classification tasks (Chapter 4). Finally, I will discuss future directions for human-AI collaboration and how to enable better two-way communication between humans and AI systems (Chapter 5).