Good emotional health is one important point to improve quality of life. On the other hand, bad emotional health can lead to social or mental health problems. To cope with the bad emotional health caused by negative emotions in daily life, we design a healthcare system that focuses on emotional aspects and provides services to improve user emotions. To improve them, we need to recognize their current emotional states. Therefore, our system integrates emotion detection to suggest appropriate services and is designed as a web-based system. While users use the system, facial expressions and speech are detected and analyzed to determine their emotions. When negative emotions are detected, our system suggests that users take a break and provides appropriate services (including relaxation, amusement and excitement services) with augmented reality and Kinect to improve their emotional states. This paper focuses on feature extraction and the classification of emotion detection by facial expressions.