In this paper, we describe the manipulation of nuts and flexible objects for assembly work by incorporating force sensorless impedance control in a robot. This robot is a dual-arm manipulator, and each of its end effectors is a three-fingered robot hand. Generally, a force sensor is used for recognition of external force on the fingertips. However, such sensors are very expensive and easily damaged by inappropriate contact with the object. This motivated us to estimate the external force without using a force sensor. In this study, the external force on the robot fingertip was estimated using a disturbance observer. Then, estimated results were compared with measurement results of the force sensor to validate the former. Impedance control was designed for a three-fingered robot hand by using this estimated external force. By application of the designed impedance control to the robot hand, the robot was successfully able to grasp nuts ranging in size from 2 mm to 12 mm through an algorithm formulated in the study. In this algorithm, we incorporated a method called following control to retain the contact of the finger with the worktable while performing the grasping task. Thus, the nut grasping task could be performed at a higher success rate, despite some errors in the robot position and vision data. Furthermore, a boiled egg considered as a flexible object was successfully grasped through force sensorless impedance control. An impedance controller for the boiled egg was designed using identified model parameters of the egg, and its effectiveness was validated experimentally.