Nowadays, we are living in the midst of a data explosion and seeing a massive growth in databases so with the wide availability of huge amounts of data; necessarily we are become in need for turning this data into useful information and knowledge, where Data mining uncovers interesting patterns and relationships hidden in a large volume of raw data and big data is a new term used to identify the datasets that are of large size and have grater complexity. The knowledge gained from data can be used for applications such as market analysis, customer retention and production control. Data mining is a massive computing task that deals with huge amount of stored data in a centralized or distributed system to extract useful information or knowledge. In this paper, we will discuss Distributed Data Mining systems, approaches, Techniques and algorithms to deal with distributed data to discover knowledge from distributed data in an effective and efficient way.