Little research has investigated the design of conversational styles of voice assistants (VA) for adults in their later adulthood with varying personalities. In this Wizard of Oz experiment, 34 middle-aged (50 to 64 years old) and 24 older adults (65 to 80 years old) participated in a user study at a simulated home, interacting with a VA using either formal or informal language. Older adults with higher agreeableness perceived VA as being more likable than middle-aged adults. Middle-aged adults showed similar technology acceptance toward the informal and formal VA, and older adults preferred using informal VA, especially those with low agreeableness. Further, while both middle-aged and older adults frequently anthropomorphized VAs by using human metaphors for them, older adults compared formal VA with professionals (e.g., librarians, teachers) and informal VA with their close ones (e.g., spouses, relatives). Overall, the conversational style showed differential effects on the perceptions of middle-aged and older adults, suggesting personalized design implications.