The purpose of the current study was to evaluate whether early postnatal stress due to cross-fostering alters the behavioral response to stress in weaned pigs. We hypothesized that piglets who were cross-fostered (FOS) into another existing litter would exhibit heightened behavioral indicators of stress after weaning in response to three behavioral stress tests compared to their non-biological litter mates (NBL), and their non-fostered biological siblings (CON). Three FOS piglets from each CON litter (n = 11 litters) were randomly selected and moved to a foster litter (n = 11 litters) 12–24 h post-farrowing, where they were nursed along NBL piglets until weaning (approximately 18 d of age). At 7- and 14-d post-weaning, all piglets underwent a novel object test (NOT) in their home pens. At 21- and 28-d post-weaning, one male and one female piglet from each treatment (CON, FOS, NBL) underwent one of two behavioral tests: social isolation and dyadic contest in an isolated 1.22 × 1.22 m novel pen. All data were analyzed using the GLIMMIX procedure in SAS. Non-biological littermate pigs exhibited a lower latency to interact with a novel object during NOT round 1 compared to CON pigs (P = 0.02), whereas FOS pigs were intermediate to but not different from either treatment (P > 0.05). A treatment by sex interaction was detected during NOT round 2 (P = 0.004), where FOS males exhibited a lower latency to interact with a novel object compared to CON pigs and FOS female pigs. Additionally, NBL females exhibited a lower latency to interaction with a novel object during NOT round 2 compared to CON, FOS female, and NBL male pigs. No treatment differences were detected during the social isolation test (P > 0.05). Treatment by sex interactions were detected during the dyadic contest for number of aggressive interactions (P < 0.01) and number of aggressive interactions initiated by an opponent (P < 0.01). Specifically, CON females and FOS males received a greater number of aggressive interactions and a greater number of aggressive interactions initiated by an opponent compared to CON males, FOS females, and NBL pigs. Taken together, these results indicate that the early postnatal stress associated with cross-fostering affects FOS piglets in a sex-specific manner. Further, NBL pigs may be the most capable of handling novel stressors post-weaning, as they exhibited fewer behavioral indicators of stress during the NOT and dyadic contests.