Characterizing the graph having the maximum or minimum spectral radius in a given class of graphs is a classical problem in spectral extremal graph theory, originally proposed by Brualdi and Solheid. Given a graph G, a vertex subset S is called a maximum generalized 4-independent set of G if the induced subgraph G[S] dose not contain a 4-tree as its subgraph, and the subset S has maximum cardinality. The cardinality of a maximum generalized 4-independent set is called the generalized 4-independence number of G. In this paper, we firstly determine the connected graph (resp. bipartite graph, tree) having the largest spectral radius over all connected graphs (resp. bipartite graphs, trees) with fixed order and generalized 4-independence number, in addition, we establish a lower bound on the generalized 4-independence number of a tree with fixed order. Secondly, we describe the structure of all the n-vertex graphs having the minimum spectral radius with generalized 4-independence number ψ, where ψ⩾⌈3n/4⌉. Finally, we identify all the connected n-vertex graphs with generalized 4-independence number ψ∈{3,⌈3n/4⌉,n−1,n−2} having the minimum spectral radius.