The conjectures of Alday et al. [Lett. Math. Phys. 91, 167–197 (2010)] and their generalizations have been mathematically formulated as the existence of an action of a W-algebra on the cohomology or K-theory of the instanton moduli space, together with a Whitaker vector [A. Braverman et al., e-print arXiv:1406.2381 (2014); D. Maulik and A. Okounkov, e-print arXiv:1211.1287 (2012), pp. 1–276; O. Schiffmann and E. Vasserot, Publ. Math. Inst. Hautes Etud. Sci. 118, 213–342 (2013)]. However, the original conjectures also predict intertwining properties with the natural higher rank version of the “Ext1 operator” which was previously studied by Okounkov and the author in Carlsson and Okounkov [Duke Math. J. 161, 1797–1815 (2012)], a result which is now sometimes referred to as AGT in rank one [A. Alba et al., Lett. Math. Phys. 98, 33–64 (2011); M. Pedrini et al., J. Geom. Phys. 103, 43–89 (2016)]. Physically, this corresponds to incorporating matter in the Nekrasov partition functions, an obviously important feature in the physical theory. It is therefore of interest to study how the Ext1 operator relates to the aforementioned structures on cohomology in higher rank, and if possible to find a formulation from which the AGT conjectures follow as a corollary. In this paper, we carry out something analogous using a modified Segal-Sugawara construction for the slˆ2C structure that appears in Nekrasov and Okounkov [Prog. Math. 244, 525–596 (2006)] for general rank. This immediately implies the AGT identities when the central charge is one, a case which is of particular interest for string theorists, and because of the natural appearance of the Seiberg-Witten curve in this setup, see, for instance, Dijkgraaf and Vafa [e-print arXiv:0909.2453 (2009).] as well as Iqbal et al. [J. High Energy Phys. 2009, 69].