A sero-epidemiology study was conducted in Dhaka, Bangladesh between January 2020 and February 2021 to assess the immune responses to ETEC infection in adults and children. (1) Background: Enterotoxigenic Escherichia coli infection is a main cause of diarrheal disease in endemic countries. The characterization of the immune responses evoked by natural infection can guide vaccine development efforts. (2) Methods: A total of 617 adult and 480 pediatric diarrheal patients were screened, and 43 adults and 46 children (below 5 years of age) with an acute ETEC infection completed the study. The plasma samples were analyzed for antibody responses against the ETEC toxins. (3) Results: Heat-stable toxin (ST)-positive ETEC is the main cause of ETEC infection in adults, unlike in children in an endemic setting. We detected very low levels of anti-ST antibodies, and no ST-neutralizing activity. However, infection with ETEC strains expressing the heat-labile toxin (LT) induced systemic antibody responses in less than 25% of subjects. The antibody levels against LTA and LTB, as well as cholera toxin (CT), correlated well. The anti-LT antibodies were shown to have LT- and CT- neutralizing activity. The antibody reactivity against linear LT epitopes did not correlate with toxin-neutralizing activity. (4) Conclusions: Unlike LT, ST is a poor antigen and even adults have low anti-ST antibody levels that do not allow for the detection of toxin-neutralizing activity.