High Time Resolution Universe Survey Research Articles

Applying Hybrid Clustering in Pulsar Candidate Sifting with Multi-modality for FAST Survey

Pulsar search is always the basis of pulsar navigation, gravitational wave detection and other research topics. Currently, the volume of pulsar candidates collected by the Five-hundred-meter Aperture Spherical radio Telescope (FAST) shows an explosive growth rate that has brought challenges for its pulsar candidate filtering system. Particularly, the multi-view heterogeneous data and class imbalance between true pulsars and non-pulsar candidates have negative effects on traditional single-modal supervised classification methods. In this study, a multi-modal and semi-supervised learning based on a pulsar candidate sifting algorithm is presented, which adopts a hybrid ensemble clustering scheme of density-based and partition-based methods combined with a feature-level fusion strategy for input data and a data partition strategy for parallelization. Experiments on both High Time Resolution Universe Survey II (HTRU2) and actual FAST observation data demonstrate that the proposed algorithm could excellently identify pulsars: On HTRU2, the precision and recall rates of its parallel mode reach 0.981 and 0.988 respectively. On FAST data, those of its parallel mode reach 0.891 and 0.961, meanwhile, the running time also significantly decreases with the increment of parallel nodes within limits. Thus, we can conclude that our algorithm could be a feasible idea for large scale pulsar candidate sifting for FAST drift scan observation.

Dealing with the Data Imbalance Problem in Pulsar Candidate Sifting Based on Feature Selection

Pulsar detection has become an active research topic in radio astronomy recently. One of the essential procedures for pulsar detection is pulsar candidate sifting (PCS), a procedure for identifying potential pulsar signals in a survey. However, pulsar candidates are always class-imbalanced, as most candidates are non-pulsars such as RFI and only a tiny part of them are from real pulsars. Class imbalance can greatly affect the performance of machine learning (ML) models, resulting in a heavy cost as some real pulsars are misjudged. To deal with the problem, techniques of choosing relevant features to discriminate pulsars from non-pulsars are focused on, which is known as feature selection. Feature selection is a process of selecting a subset of the most relevant features from a feature pool. The distinguishing features between pulsars and non-pulsars can significantly improve the performance of the classifier even if the data are highly imbalanced. In this work, an algorithm for feature selection called the K-fold Relief-Greedy (KFRG) algorithm is designed. KFRG is a two-stage algorithm. In the first stage, it filters out some irrelevant features according to their K-fold Relief scores, while in the second stage, it removes the redundant features and selects the most relevant features by a forward greedy search strategy. Experiments on the data set of the High Time Resolution Universe survey verified that ML models based on KFRG are capable of PCS, correctly separating pulsars from non-pulsars even if the candidates are highly class-imbalanced.

The Southern-sky MWA Rapid Two-metre (SMART) pulsar survey—II. Survey status, pulsar census, and first pulsar discoveries

Abstract In Paper I, we presented an overview of the Southern-sky MWA Rapid Two-metre (SMART) survey, including the survey design and search pipeline. While the combination of MWA’s large field-of-view and the voltage capture system brings a survey speed of ${\sim} 450\, {\textrm{deg}}^{2}\,\textrm{h}^{-1}$ , the progression of the survey relies on the availability of compact configuration of the Phase II array. Over the past few years, by taking advantage of multiple windows of opportunity when the compact configuration was available, we have advanced the survey to 75% of the planned sky coverage. To date, about 10% of the data collected thus far have been processed for a first-pass search, where 10 min of observation is processed for dispersion measures out to 250 ${\textrm{pc cm}}^{-3}$ , to realise a shallow survey that is largely sensitive to long-period pulsars. The ongoing analysis has led to two new pulsar discoveries, as well as an independent discovery and a rediscovery of a previously incorrectly characterised pulsar, all from ${\sim} 3\% $ of the data for which candidate scrutiny is completed. In this sequel to Paper I, we describe the strategies for further detailed follow-up including improved sky localisation and convergence to timing solution, and illustrate them using example pulsar discoveries. The processing has also led to re-detection of 120 pulsars in the SMART observing band, bringing the total number of pulsars detected to date with the MWA to 180, and these are used to assess the search sensitivity of current processing pipelines. The planned second-pass (deep survey) processing is expected to yield a three-fold increase in sensitivity for long-period pulsars, and a substantial improvement to millisecond pulsars by adopting optimal de-dispersion plans. The SMART survey will complement the highly successful Parkes High Time Resolution Universe survey at 1.2–1.5 GHz, and inform future large survey efforts such as those planned with the low-frequency Square Kilometre Array (SKA-Low).

A Preliminary Study of Large Scale Pulsar Candidate Sifting Based on Parallel Hybrid Clustering

Pulsar candidate sifting is an essential part of pulsar analysis pipelines for discovering new pulsars. To solve the problem of data mining of a large number of pulsar data using a Five-hundred-meter Aperture Spherical radio Telescope (FAST), a parallel pulsar candidate sifting algorithm based on semi-supervised clustering is proposed, which adopts a hybrid clustering scheme based on density hierarchy and the partition method, combined with a Spark-based parallel model and a sliding window-based partition strategy. Experiments on the two datasets, HTRU (The High Time-Resolution Universe Survey) 2 and AOD-FAST (Actual Observation Data from FAST), show that the algorithm can excellently identify the pulsars with high performance: On HTRU2, the Precision and Recall rates are 0.946 and 0.905, and those on AOD-FAST are 0.787 and 0.994, respectively; the running time on both datasets is also significantly reduced compared with its serial execution mode. It can be concluded that the proposed algorithm provides a feasible idea for astronomical data mining of FAST observation.

Adaboost-DSNN: an adaptive boosting algorithm based on deep self normalized neural network for pulsar identification

ABSTRACT A modern pulsar survey generates a large number of pulsar candidates. Filtering these pulsar candidates in a large astronomical data set is an important step towards discovering new pulsars. In this paper, a novel adaptive boosting algorithm based on deep self normalized neural network (Adaboost-DSNN) is proposed to accurately classify pulsar and non-pulsar signals. To train the proposed method on a highly imbalanced data set, the Synthetic Minority Oversampling TEchnique (SMOTE) was initially employed for balancing the data set. Then, a deep ensemble network combined with a deep self-normalized neural network and adaptive boosting was developed to train and learn the processed pulsar data. The design of the proposed Adaboost-DSNN method significantly reduced the computational time when dealing with large astronomical data sets, while also improving the classification performance. The scaled exponential liner units activation function was used to normalize the data. Considering their neighbour information and the special dropout technique (α-dropout), Adaboost-DSNN displayed good pulsar classification performance, while preserving the data properties across subsequent layers. The proposed Adaboost-DSNN method was tested on the High Time Resolution Universe Survey data sets (HTRU-1 and HTRU-2). According to experimental results, Adaboost-DSNN outperform other state-of-the-art methods with respect to training time and F1-score. The training time of the Adaboost-DSNN model is 10x times faster compared to other models of this kind.

Research on Pulsar Candidate Identification Method Based on Deep Residual Neural Network

As the next-generation radio astronomical telescopes continuously improve and develop, the pulsar survey will produce millions of pulsar candidates, which pose considerable challenges for pulsar identification and classification. The rapidly evolving artificial intelligence (AI) techniques are being used for pulsar identification and discovery of new pulsars. Using the pulsar data set observed with the Parkes telescope, namely the High Time Resolution Universe Survey (HTRUS), a 14-layer deep residual network has been designed (called the Residual Network, ResNet) for pulsar candidate classifications. In the HTRUS sample data, the number of non-pulsar candidates (i.e., negative samples) is much larger than that of pulsar candidates (i.e., positive samples). The imbalance between the positive and negative samples is prone to result in model misjudgement. By using the over-sampling technique to enhance the data of positive samples in the training set and adjusting the ratio of positive and negative samples, we have solved this imbalance problem. In the training process, the hyperparameters are adjusted by means of 5-fold cross validation to build the model. The test results indicate that the model can achieve a high precision (98%) and recall (100%), the F1-score can reach 99%, and that the implementation of each sample test needs only 7 ms, it has provided a feasible approach for the future big-data analysis of pulsars.

The High Time Resolution Universe Pulsar Survey – XV. Completion of the intermediate-latitude survey with the discovery and timing of 25 further pulsars

We report on the latest six pulsars discovered through our standard pipeline in the intermediate-latitude region (|b| < 15 deg) of the Parkes High Time Resolution Universe Survey (HTRU). We also present timing solutions for the new discoveries and for 19 further pulsars for which only discovery parameters were previously published. Highlights of the presented sample include the isolated millisecond pulsar J1826-2415, the long-period binary pulsar J1837-0822 in a mildly eccentric 98-day orbit with a > 0.27 M_sun companion, and the nulling pulsar J1638-4233, detected only 10% of the time. Other interesting objects are PSR J1757-1500, exhibiting sporadic mode changes, and PSR J1635-2616 showing one glitch over 6 years. The new discoveries bring the total count of HTRU intermediate-latitude pulsars to 113, 25% of which are recycled pulsars. This is the higest ratio of recycled over ordinary pulsars discoveries of all recent pulsar surveys in this region of the sky. Among HTRU recycled pulsars, four are isolated objects. Comparing the characteristics of Galactic fully-recycled isolated MSPs with those of eclipsing binaries ('spiders'), from which the former are believed to have formed, we highlight a discrepancy in their spatial distribution. This may reflect a difference in the natal kick, hence, possibly, a different formation path. On the other hand, however, isolated fully-recycled MSPs spin periods are, on average, longer than those of spiders, in line with what one would expect, from simple magnetic-dipole spin-down, if the former were indeed evolved from the latter.

The High Time Resolution Universe survey – XIV. Discovery of 23 pulsars through GPU-accelerated reprocessing

We have performed a new search for radio pulsars in archival data of the intermediate and high Galactic latitude parts of the Southern High Time Resolution Universe pulsar survey. This is the first time the entire dataset has been searched for binary pulsars, an achievement enabled by GPU-accelerated dedispersion and periodicity search codes nearly 50 times faster than the previously used pipeline. Candidate selection was handled entirely by a Machine Learning algorithm, allowing for the assessment of 17.6 million candidates in a few person-days. We have also introduced an outlier detection algorithm for efficient radio-frequency interference (RFI) mitigation on folded data, a new approach that enabled the discovery of pulsars previously masked by RFI. We discuss implications for future searches, particularly the importance of expanding work on RFI mitigation to improve survey completeness. In total we discovered 23 previously unknown sources, including 6 millisecond pulsars and at least 4 pulsars in binary systems. We also found an elusive but credible redback candidate that we have yet to confirm.

Pulsar candidate recognition with deep learning

In this paper, we present a deep learning-based recognition algorithm to identify pulsars by observing data containing millions of candidates including radio frequency interference and noise sources. The dataset is obtained from the High Time Resolution Universe survey created and updated by the Parkes telescope. We investigate several effective single and combined features via simple logistic regression. To deal with the imbalanced dataset, we oversimplify the original dataset at different sampling rates, which is also one of the learning parameters. After training the pre-processed dataset via a convolutional neural network, we provide a cross-validated evaluation of all candidates. Results show that the deep-learning based recognition algorithm can identify the pulsar and radio frequency interference signals with high accuracy. The precision and recall of radio frequency interference are both 100%, and those of pulsars are 91% and 94%, respectively.

Study of the eclipse region of the redback millisecond pulsar J1431–4715

We report on the rotational, astrometric and orbital parameters for PSR J1431–4715, and we also present a preliminary analysis of the eclipsing region. This pulsar was discovered in the High Time Resolution Universe survey and it belongs to the class of “redback” systems. The minimum estimated mass for the companion of J1431–4715 is, indeed, 0.13 M⊙. Thanks to multi-frequency observations, obtained at the 64 m Parkes radio telescope, we note that the magnitude and the duration of the eclipse delay depend upon the observing frequency.

PSR J2322−2650 – a low-luminosity millisecond pulsar with a planetary-mass companion

We present the discovery of a binary millisecond pulsar (MSP), PSR J2322$-$2650, found in the Southern section of the High Time Resolution Universe survey. This system contains a 3.5-ms pulsar with a $\sim10^{-3}$ M$_{\odot}$ companion in a 7.75-hour circular orbit. Follow-up observations at the Parkes and Lovell telescopes have led to precise measurements of the astrometric and spin parameters, including the period derivative, timing parallax, and proper motion. PSR J2322$-$2650 has a parallax of $4.4\pm1.2$ mas, and is thus at an inferred distance of $230^{+90}_{-50}$ pc, making this system a candidate for optical studies. We have detected a source of $R\approx26.4$ mag at the radio position in a single $R$-band observation with the Keck Telescope, and this is consistent with the blackbody temperature we would expect from the companion if it fills its Roche lobe. The intrinsic period derivative of PSR J2322$-$2650 is among the lowest known, $4.4(4)\times10^{-22}$ s s$^{-1}$, implying a low surface magnetic field strength, $4.0(4)\times10^7$ G. Its mean radio flux density of 160 $\mu$Jy combined with the distance implies that its radio luminosity is the lowest ever measured, $0.008(5)$ mJy kpc$^2$. The inferred population of these systems in the Galaxy may be very significant, suggesting that this is a common MSP evolutionary path.

A survey of FRB fields: limits on repeatability

Several theories exist to explain the source of the bright, millisecond duration pulses known as fast radio bursts (FRBs). If the progenitors of FRBs are non-cataclysmic, such as giant pulses from pulsars, pulsar–planet binaries, or magnetar flares, FRB emission may be seen to repeat. We have undertaken a survey of the fields of eight known FRBs from the High Time Resolution Universe survey to search for repeating pulses. Although no repeat pulses were detected the survey yielded the detection of a new FRB, described in Petroff et al. (2015a). From our observations we rule out periodic repeating sources with periods P ≤ 8.6 h and rule out sources with periods 8.6 < P < 21 h at the 90 per cent confidence level. At P ≥ 21 h our limits fall off as ∼1/P. Dedicated and persistent observations of FRB source fields are needed to rule out repetition on longer time-scales, a task well-suited to next generation wide-field transient detectors.

On the paucity of fast radio bursts at low Galactic latitudes

We examine the effect of Galactic diffractive interstellar scintillation as a means of explaining the reported deficit of fast radio burst (FRB) detections at low Galactic latitude. We model the unknown underlying FRBflux density distribution as a power lawwith a rate scaling as S−5/2+δ ν and account for the fact that the FRBs are detected at unknown positions within the telescope beam. We find that the event rate of FRBs located off the Galactic plane may be enhanced by a factor of ∼30–300 per cent relative to objects near the Galactic plane without necessarily affecting the slope of the distribution. For FRBs whose flux densities are subject to relatively weak diffractive scintillation, as is typical for events detected at high Galactic latitudes, we demonstrate that an effect associated with Eddington bias is responsible for the enhancement. The magnitude of the enhancement increases with the steepness of the underlying flux density distribution, so that existing limits on the disparity in event rates between high and lowGalactic latitudes suggest that the FRB population has a steep differential flux density distribution, scaling as S−3.5 ν or steeper. Existing estimates of the event rate in the flux density range probed by the High Time Resolution Universe survey overestimate the true rate by a factor of ∼3.

The High Time Resolution Universe survey – XI. Discovery of five recycled pulsars and the optical detectability of survey white dwarf companions

We present the discovery of a further five recycled pulsar systems in the mid-Galactic latitude portion of the High Time Resolution Universe (HTRU) Survey. The pulsars have rotational periods ranging from 2 ms to 66 ms, and four are in binary systems with orbital periods between 10.8 hours and 9.0 days. Three of these binary systems are particularly interesting; PSR J1227-6208 has a pulse period of 34.5 ms and the highest mass function of all pulsars with near-circular orbits. The circular orbit suggests that the companion is not another neutron star, so future timing experiments may reveal one of the heaviest white dwarfs ever found ($>$ 1.3 M$_\odot$). Timing observations of PSR J1431$-$4715 indicate that it is eclipsed by its companion which has a mass indicating it belongs to the redback class of eclipsing millisecond pulsars. PSR J1653-2054 has a companion with a minimum mass of only $0.08$ M$_\odot$, placing it among the class of pulsars with low-mass companions. Unlike the majority of such systems, however, no evidence of eclipses is seen at 1.4 GHz.

AN ABSENCE OF FAST RADIO BURSTS AT INTERMEDIATE GALACTIC LATITUDES

Fast radio bursts (FRBs) are an emerging class of bright, highly dispersed radio pulses. Recent work by Thornton et al. (2013) has revealed a population of FRBs in the High Time Resolution Universe (HTRU) survey at high Galactic latitudes. A variety of progenitors have been proposed including cataclysmic events at cosmological distances, Galactic flare stars, and terrestrial radio frequency interference. Here we report on a search for FRBs at intermediate Galactic latitudes ($-15^{\circ}$ $< b <$ 15$^{\circ}$) in data taken as part of the HTRU survey. No FRBs were discovered in this region. Several effects such as dispersion, scattering, sky temperature and scintillation decrease the sensitivity by more than 3$\sigma$ in $\sim$20\% of survey pointings. Including all of these effects, we exclude the hypothesis that FRBs are uniformly distributed on the sky with 99\% confidence. This low probability implies that additional factors -- not accounted for by standard Galactic models -- must be included to ease the discrepancy between the detection rates at high and low Galactic latitudes. A revised rate estimate or another strong and heretofore unknown selection effect in Galactic latitude would provide closer agreement between the surveys' detection rates. The dearth of detections at low Galactic latitude disfavors a Galactic origin for these bursts.

The Northern High Time Resolution Universe pulsar survey – I. Setup and initial discoveries

Abstract We report on the setup and initial discoveries of the Northern High Time Resolution Universe survey for pulsars and fast transients, the first major pulsar survey conducted with the 100-m Effelsberg radio telescope and the first in 20 years to observe the whole northern sky at high radio frequencies. Using a newly developed 7-beam receiver system combined with a state-of-the-art polyphase filterbank, we record an effective bandwidth of 240 MHz in 410 channels centred on 1.36 GHz with a time resolution of 54 μs. Such fine time and frequency resolution increases our sensitivity to millisecond pulsars and fast transients, especially deep inside the Galaxy, where previous surveys have been limited due to intrachannel dispersive smearing. To optimize observing time, the survey is split into three integration regimes dependent on Galactic latitude, with 1500, 180 and 90-s integrations for latitude ranges |b| &amp;lt; 3 $_{.}^{\circ}$5, |b| &amp;lt; 15° and |b| &amp;gt; 15°, respectively. The survey has so far resulted in the discovery of 15 radio pulsars, including a pulsar with a characteristic age of ∼18 kyr, PSR J2004+3429, and a highly eccentric, binary millisecond pulsar, PSR J1946+3417. All newly discovered pulsars are timed using the 76-m Lovell radio telescope at the Jodrell Bank Observatory and the Effelsberg radio telescope. We present timing solutions for all newly discovered pulsars and discuss potential supernova remnant associations for PSR J2004+3429.

The High Time Resolution Universe Pulsar Survey –VIII. The Galactic millisecond pulsar population

We have used millisecond pulsars (MSPs) from the southern High Time Resolution Universe (HTRU) intermediate latitude survey area to simulate the distribution and total population of MSPs in the Galaxy. Our model makes use of the scale factor method, which estimates the ratio of the total number of MSPs in the Galaxy to the known sample. Using our best fit value for the z-height, z=500 pc, we find an underlying population of MSPs of 8.3(\pm 4.2)*10^4 sources down to a limiting luminosity of L_min=0.1 mJy kpc^2 and a luminosity distribution with a steep slope of d\log N/d\log L = -1.45(\pm 0.14). However, at the low end of the luminosity distribution, the uncertainties introduced by small number statistics are large. By omitting very low luminosity pulsars, we find a Galactic population above L_min=0.2 mJy kpc^2 of only 3.0(\pm 0.7)*10^4 MSPs. We have also simulated pulsars with periods shorter than any known MSP, and estimate the maximum number of sub-MSPs in the Galaxy to be 7.8(\pm 5.0)*10^4 pulsars at L=0.1 mJy kpc^2. In addition, we estimate that the high and low latitude parts of the southern HTRU survey will detect 68 and 42 MSPs respectively, including 78 new discoveries. Pulsar luminosity, and hence flux density, is an important input parameter in the model. Some of the published flux densities for the pulsars in our sample do not agree with the observed flux densities from our data set, and we have instead calculated average luminosities from archival data from the Parkes Telescope. We found many luminosities to be very different than their catalogue values, leading to very different population estimates. Large variations in flux density highlight the importance of including scintillation effects in MSP population studies.

The High Time Resolution Universe Pulsar Survey — VI. An artificial neural network and timing of 75 pulsars

We present 75 pulsars discovered in the mid-latitude portion of the High Time Resolution Universe survey, 54 of which have full timing solutions. All the pulsars have spin periods greater than 100 ms, and none of those with timing solutions are in binaries. Two display particularly interesting behaviour; PSR J1054-5944 is found to be an intermittent pulsar, and PSR J1809-0119 has glitched twice since its discovery. In the second half of the paper we discuss the development and application of an artificial neural network in the data-processing pipeline for the survey. We discuss the tests that were used to generate scores and find that our neural network was able to reject over 99% of the candidates produced in the data processing, and able to blindly detect 85% of pulsars. We suggest that improvements to the accuracy should be possible if further care is taken when training an artificial neural network; for example ensuring that a representative sample of the pulsar population is used during the training process, or the use of different artificial neural networks for the detection of different types of pulsars.

Conducting the deepest all-sky pulsar survey ever: the all-sky High Time Resolution Universe survey

AbstractThe extreme conditions found in and around pulsars make them fantastic natural laboratories, providing insights to a rich variety of fundamental physics and astronomy. To discover more pulsars we have begun the High Time Resolution Universe (HTRU) survey: a blind survey of the northern sky with the 100-m Effelsberg radio telescope in Germany and a twin survey of the southern sky with the 64-m Parkes radio telescope in Australia. The HTRU is an international collaboration with expertise shared among the MPIfR in Germany, ATNF/CASS and Swinburne University of Technology in Australia, University of Manchester in the UK and INAF in Italy. The HTRU survey uses multi-beam receivers and backends constructed with recent advancements in technology, providing unprecedentedly high time and frequency resolution, allowing us to probe deeper into the Galaxy than ever before. While a general overview of HTRU has been given by Keith at this conference, here we focus on three further aspects of HTRU discoveries and highlights. These include the ‘Diamond-planet pulsar’ binary J1719-1438 and a second similar system recently discovered. In addition, we provide specifications of the HTRU-North survey and an update of its status. In the last section we give an overview of the search for highly-accelerated binaries in the Galactic plane region. We discuss the computational challenges arising from the processing of the petabyte-sized HTRU survey data. We present an innovative segmented search technique which aims to increase our chances of discovering highly accelerated relativistic binary systems, potentially including pulsar-black-hole binaries.

The High Time Resolution Universe surveys for pulsars and fast transients

AbstractThe High Time Resolution Universe survey for pulsars and transients is the first truly all-sky pulsar survey, taking place at the Parkes Radio Telescope in Australia and the Effelsberg Radio Telescope in Germany. Utilising multibeam receivers with custom built all-digital recorders the survey targets the fastest millisecond pulsars and radio transients on timescales of 64 μs to a few seconds. The new multibeam digital filter-bank system at has a factor of eight improvement in frequency resolution over previous Parkes multibeam surveys, allowing us to probe further into the Galactic plane for short duration signals. The survey is split into low, mid and high Galactic latitude regions. The mid-latitude portion of the southern hemisphere survey is now completed, discovering 107 previously unknown pulsars, including 26 millisecond pulsars. To date, the total number of discoveries in the combined survey is 135 and 29 MSPs These discoveries include the first magnetar to be discovered by it's radio emission, unusual low-mass binaries, gamma-ray pulsars and pulsars suitable for pulsar timing array experiments.