Thermoresponsive hydrogel-based actuators have shown great potential in tissue engineering, biosensors and soft robots, while the preparation of temperature-driven bilayer hydrogel actuators with rapid responseness and recovery properties remains a challenge. Inspired by the layered heterostructure of human skin, we developed a Janus bilayer structure actuator consisting of thermoresponsive double network (TDN) hydrogel and polydimethylsiloxane (PDMS) elastomer with tough interface adhesion. Specifically, PNaAMPS/P(NIPAM-co-AAm) TDN hydrogel via photocopolymerization of N-isopropyl acrylamide (NIPAM) and acrylamide (AAm) and reinforced by microgel poly(2-acrylamide-2-methylpropane sulfonate sodium salt) (PNaAMPS) was in situ grafted onto benzophenone-activated PDMS surface, exhibiting strong interface adhesion (628 Jm-2 adhesion energy). The thermoresponsive shape deformations of the double-layer structure actuator were controllable by adjusting hydrogel/elastomer thickness ratio and introduction of metal ions, taking advantage of the different swelling and shrinkage characteristics of the hydrogel layer and the elastomer layer. The TDN/PDMS bilayer structure actuator, which integrates the "Janus" structure, thermal responsiveness and photothermal conversion, provides new ideas for the application and remote control of soft robots and soft actuators.