Well dispersed polyethylene (PE) nanocomposites were developed. Montmorillonite (MMT) as aluminosilicate clay layers was modified using organic silanes of different side chains. The MMT was grafted using 3-(trimethoxysilyl)propylamine, N-[3-(trimethoxysilyl)propyl]ethylenediamine, and trimethoxyvinylsilane. The modification process of MMT using organic different silanes was elucidated using microscopic, thermogravimetric, spectroscopic, and X-ray diffraction tools. Then, the developed organoclays were dispersed uniformly in PE matrix producing well exfoliated and dispersed polymer nanocomposites. The mass loadings of dispersed organoclay were varied and the impact of organic silane structure was studied. Thermal stability and flammability properties of the developed polymer nanocomposites were evaluated. The peak heat release rate and average heat release rate were reduced by 48% and 61%, respectively compared to virgin polymer. Also, the average mass loss rate was significantly reduced by 50%. This is in addition to reduction in emission of carbon monoxide (CO) and carbon dioxide (CO2) by 45% and 56%, respectively. The effect of side chain of organosilane on the performance of modified clay layers was studied. The toxicity of gases evolved during combustion process of PE and their polymer nanocomposites were evaluated using Fourier transform infrared connected to cone calorimeter. Additionally, the influence of organic silane on the pyrolysis and toxic gases emission was further studied.