This study explored the visual images that affect middle-aged and elderly consumer groups in organic food packaging design, and summarized the happiness of visual design elements for organic food as a reference for design. This study used expert interviews, the KJ method to collect image vocabulary and select representative samples, and the Semantic Differential method (SD method) to evaluate the happiness image of organic food. Then, the questionnaire survey method was used to investigate the subjects’ attention regarding happiness in the visual design elements of organic food. This study adopted one-way ANOVA to understand the differences of consumers’ visual images for different organic food packages. The results show that the living patterns of the consumer groups in organic food packaging design can be classified into four groups. The “Self-actualization Group” thinks that happiness organic food packaging is healthy, reassuring, environment-friendly, healing, comfortable, and novel. The “Caring for Health Group” believes that happiness organic food packaging is healthy, reassuring, and environmentally friendly. The “Lohas Food Safety Group” believes that happiness organic food is packaged in a healthy, simple, and environment-friendly way. The “Extroverted Loyalty Group” believes that happiness organic food packaging is healthy, comfortable, professional, simple, environment-friendly, and novel.