Aggregation of mutant forms of Huntingtin is the underlying feature of neurodegeneration observed in Huntington's disorder. In addition to neurons, cellular processes in non-neuronal cell types are also shown to be affected. Cells expressing neurodegeneration-associated mutant proteins show altered uptake of ligands, suggestive of impaired endocytosis, in a manner as yet unknown. Using live cell imaging, we show that clathrin-mediated endocytosis (CME) is affected in Drosophila hemocytes and mammalian cells containing Huntingtin aggregates. This is also accompanied by alterations in the organization of the actin cytoskeleton resulting in increased cellular stiffness. Further, we find that Huntingtin aggregates sequester actin and actin-modifying proteins. Overexpression of Hip1 or Arp3 (actin-interacting proteins) could restore CME and cellular stiffness in cells containing Huntingtin aggregates. Neurodegeneration driven by pathogenic Huntingtin was also rescued upon overexpression of either Hip1 or Arp3 in Drosophila. Examination of other pathogenic aggregates revealed that TDP-43 also displayed defective CME, altered actin organization and increased stiffness, similar to pathogenic Huntingtin. Together, our results point to an intimate connection between dysfunctional CME, actin misorganization and increased cellular stiffness caused by alteration in the local intracellular environment by pathogenic aggregates.