Better Measurement or Larger Samples? Data Collection for Policy Learning with Unobserved Heterogeneity

Key Factors for Implementing Supported Employment

PurposeThis study aimed to develop and test the impact of a social media-based intervention for improving the business skills and income of young smallholder farmers.Design/methodology/approachFirst, the researchers used an exploratory approach to develop a social media-based intervention for acquiring business skills and improving income. Second, the researchers tested the effectiveness of the developed programme on a sample of 506 young smallholder farmers. Finally, the intervention and data collection took place over five years (2017–2021).FindingsThe result showed steady improvements in business skills and income from 2017 to 2021 for the treatment group, unlike the control group. Also, improvements in business skills led to a reduction in expenses and an increase in profit from 2017 to 2021. A further evaluation of the result showed that an addition of 5.1 mean scores in business skills led to the addition of $91 income between 2017 and 2018; for 2018–2019, 2.6 improvements in business skills increased income by $123. For 2019–2020, a 2.7 improvement increased income by $209, whilst for 2020–2021, a 1.6 improvement increased income by $320.Originality/valueThe results of this study could help explore ways of using social media to change behaviour aimed at improving income amongst young smallholder farmers.

Introduction The detection, characterization, and communication of information about drug toxicity are integral to medicine and drug regulation in all medical fields. Although the challenges we face in HIV may appear uniquely daunting, they are not essentially different. Over the first two decades of the HIV epidemic, the importance of drug safety monitoring was overshadowed by the need to develop potent therapies capable of arresting a fatal disease process. It is now clear, however, that the treatment successes achieved mainly through highly active antiretroviral therapy (HAART) are tempered significantly by drug toxicity [1–3]. These toxicities often occur in patients who have been exposed to multiple drugs for prolonged periods of time, thus the monitoring of long-term toxicities necessitates efforts extending beyond the usual follow-up of many cohorts and nearly all clinical trials. This review focuses on drug safety monitoring of HIV treatments and, in particular, the strengths and limitations of the available approaches for detecting and characterizing long-term toxicities. To understand the challenges specific to the monitoring of long-term toxicities, it is critical to understand the interconnected functions of both pharmacovigilance and pharmacoepidemiology. The purpose of pharmacovigilance is to detect previously unknown adverse drug effects. Pharmacovigilance sets the stage for formal pharmacoepidemiology studies, which involve control groups and are meant to refute or confirm and quantify drug safety risks. Risk management follows. Findings are communicated to patients and providers, while governments, manufacturers, and professional groups devise ways to change medical practice in order to avoid further toxicities. These processes are illustrated schematically in Figure 1. (A broader definition places all steps under the umbrella of pharmacovigilance, but in this review, we will refer to pharmacovigilance and pharmacoepidemiology as distinct concepts).Fig. 1.: Process of pharmacovigilance and pharmacoepidemiology.From a regulatory standpoint, safety monitoring of pharmaceuticals occurs in two phases – (1) before and (2) after a drug is approved. Leading up to approval, clinical trials provide data on predefined efficacy questions, frequent adverse events and immediate safety. Following approval, during postmarketing surveillance, safety information is derived primarily from spontaneous reporting systems supported by regulatory authorities in every industrialized country; further information may be generated by large post-marketing (Phase III and IV) clinical studies. The current system of drug safety assessment faces significant constraints, many of which are explained below. Despite these challenges, enhanced understanding of toxicities associated with use of antiretroviral medications can be expected to improve HIV care in several ways. Patients will be provided with information leading to more accurate expectations, perhaps decreasing disappointment and frustration when chronic, low-grade toxicities occur [4]. More realistic expectations may increase a feeling of trust and teamwork between patient and provider, which could lead to greater adherence [5]. Furthermore, better evidence regarding long-term toxicities will improve advice given to patients by clinicians about timing of initial therapy, choice of regimen, and drug substitutions or discontinuations. Better safety data may also lead to insights into the mechanism of toxicity and can eventually assist in drug development, in screening patients at high risk for toxicity, and in developing useful strategies for toxicity monitoring and management. For the purposes of this review, the term adverse event (AE) is defined as any untoward medical occurrence in a subject who has been administered a pharmaceutical product. The drug may or may not be causally related to the AE – in the cases where it is, the event is termed an adverse drug reaction (ADR). Indeed, sorting out the role of the drug in causing the AE is the goal of pharmacoepidemiology and drug safety monitoring. Available systems for monitoring the safety of HIV treatments – signal detection and hypothesis testing Signal detection Several systems currently are available for the detection of HIV-related AEs. They include pre- and post-approval clinical trials, spontaneous reporting systems such as the US Food and Drug Administration's (FDA) Adverse Event Reporting System and similar arrangements in the European Union and other industrialized countries, automated databases such as those compiled by Health Maintenance Organizations (HMOs), and observational cohorts. Randomized clinical trials Because individuals are first exposed to pharmaceutical products in the setting of clinical trials, these trials comprise a potentially valuable early source of toxicity data especially for relatively common ADRs with short-term onset. An example of one such toxicity is abacavir hypersensitivity reaction, which affects approximately 5% of individuals given the drug [6]. Randomized clinical trials (RCTs), however, have several limitations as signal generators. Pre-marketing studies usually recruit small, homogenous patient populations for study – typically fewer than 3000 exposed individuals. In this case, even if no serious ADRs are detected, one can only be confident that the rate of events is not greater than 3 per 3000, often referred to as the 'rule of 3' [7]. Thus, detection of relatively infrequent ADRs such as lactic acidosis is unlikely. Furthermore, individuals with significant comorbidities (e.g., hepatitis B and C) are commonly excluded, making detection of ADRs in these groups impossible. Similarly, ADRs may occur only in selected patient subsets that are insufficiently recruited and included in these trials. For example, female sex is associated with development of cutaneous rash caused by nevirapine [8]. Trials carried out largely in a male population could potentially miss or at least underestimate the occurrence of this reaction. Similarly, under-representation of ethnic minorities in clinical trials adds to this problem. Another limitation of RCTs (both pre- and post-approval) for the detection of long-term ADRs in HIV is the relatively short follow-up time of clinical trials. Indeed, increasing the follow-up of RCTs increases the cost and complexity of these studies. Moreover, the validity of the results often diminishes as losses to follow-up increase. However, several trials with extended follow-up are currently ongoing, including ACTG 384, the FIRST study, INITIO, and SMART, demonstrating that although difficult, long-term RCTs are feasible. Yet another limitation is the possibility that certain ADRs occurring in RCTs are under-reported. 'Minor' ADRs such as mood disorders or sleep disturbances may go undetected unless specifically targeted in data collection. These low-grade toxicities, however, may have substantial effects on adherence, which has been shown to affect several HIV-related outcomes [9,10]. Other ADRs, such as lipodystrophy, may not be classified in a standardized way, leading to poor detection in RCTs [11]. Homogenous patient populations, small sample sizes, and short durations of follow-up therefore significantly limit the use of RCTs for the detection of uncommon and late-onset toxicities. Spontaneous reporting systems After approval, the major mechanism of post-marketing surveillance is spontaneous reporting systems. This mechanism, used in both the United States and Europe, helps to identify ADRs that may not have been revealed during pre-approval trials. Completely passive in design, the identification of ADRs begins with the collection of spontaneous AE reports made by health professionals and patients and is typified by the FDA's MedWatch system [12]. Major strengths are the large scale (indeed, they potentially include all patients in clinical care in those countries with these systems) and the diversity of the population potentially included over long periods of time. Significant limitations do, however, exist. In the US, no federal laws or regulations require health care providers to report AEs related to pharmaceuticals, and it is estimated that the FDA receives reports on less than 1% of suspected serious drug-related events [13]. In many cases, physicians may feel an event is too trivial or too well known to report [14]. Other reasons for under-reporting may include physician guilt about harming a patient, fear of potential litigation, ambition to collect and publish cases, lack of awareness that an ADR has occurred, and lack of knowledge of how to report AEs via the available system [15]. A further limitation probably relates to lack of time and/or unwillingness to become involved in follow-up documentation or verification of reported AEs. Because of these limitations, the actual number of patients with a particular AE (the numerator of the AE rate) is unknown. Once reports are received, AEs are grouped into aggregate categories (i.e. a 'rash' may be further separated into 'maculopapular rash' or 'bullous eruption', etc.) based on standardized medical terminology dictionaries, such as MedDRA (Medical Dictionary for Drug Regulatory Activities) [16]. Some AE reports, however, may not be easily assigned to a single specific category. Moreover, terminology used in standardized medical dictionaries, despite ongoing efforts at harmonization, is often not consistent across international systems, adding further complexity to the process. After categorization, further challenges arise from the need to systematically identify and characterize those AEs that are observed to a greater rate than expected. The process of identification, based on pattern recognition, involves the use of prior knowledge and scientific inference to separate consistent, replicable 'signals' from a background of database 'noise' [17]. Given that any large surveillance system will produce many interesting but perhaps biased patterns of AEs and disease, it becomes vital that carefully reviewed associations be followed by formal pharmacoepidemiology studies in order to further evaluate cause and effect. This is particularly true in the case of HIV, where causal associations between drug and AE are often complicated by the multiplicity of treatments, any one of which might account for the toxicity, and by the possibility that the disease itself, apart from any treatment, may be the culprit. Indeed, the process of signal detection produces case reports and case series; observational and/or interventional studies that utilize control groups are critical in order to formally define risk. Automated databases Automated databases, originally developed to support computerized billing systems, are another potential resource for detecting antiretroviral toxicity. Automated databases provide large numbers of patients followed longitudinally through various health-care encounters and sequences of drug use. Necessarily smaller in their population coverage than nationwide spontaneous reporting systems, they nonetheless offer more complete ascertainment of serious events in the persons included in the database. Furthermore, some large systems (most notably in the UK, but also found in continental Europe and in the United States) were initially created as computerized medical records, and may, depending on local privacy laws, be used for individual record review or for population studies. The most common uses of these data historically have been to provide quantitative evaluation of signals generated elsewhere. They also provide a strong platform for building active surveillance systems. Another strength of automated databases is the relative heterogeneity of patients exposed to drug. Furthermore, prescription information may be available, providing one way of assessing duration of drug exposure. The longitudinal nature of the data is particularly valuable when the goal is to detect long-term toxicities. A partial list of automated databases available for the detection of ADRs in HIV is given in Table 1. Note that certain databases not included in the table, such as the General Practice Research Database in the UK and the Saskatchewan database in Canada, although large, currently have limited utility in HIV due to their relatively small numbers of HIV-infected patients included. A helpful discussion of the use of these specific databases for pharmacoepidmiologic purposes is contained in Part III of the book Pharmacoepidemiology [7].Table 1: Automated databases available for study of adverse events in HIV.However, in order for an AE to be coded in an automated database it must be recognized. The tendency of providers to recognize AEs as ADRs may in turn relate to diagnostic suspicion and/or other biases that over or underestimate the true association of a drug and a specific toxicity. In some cases, AEs may not be recognized at all. All of these issues may limit the ability to accurately identify and study toxicities using these sources. In some cases, billing codes (e.g. International Classification of Disease (ICD) codes) may capture the events with adequate sensitivity and specificity. However, the target event might be spread across many codes (e.g. upper gastrointestinal bleeding could be coded as upper gastrointestinal bleeding not otherwise specified, hematemesis, melena, or acute duodenal ulcer with bleeding), be buried under a rubric that contains numerous other entities, or correspond to an evolving syndrome for which no code yet exists, such as lipodystrophy syndrome [11]. Case ascertainment in this setting often requires multiple different aggregations of codes as different definitions of the same disease [18]. Chart review of suspected cases is almost always required. Alternatively, if linkage to the medical record is possible, some ADRs, such as anemia, may be ascertained via laboratory data. In general, those clinical entities that bring patients to the attention of caregivers, result in a quick and coded diagnosis, and can be supported or confirmed by a laboratory test (or chart review) are candidates for study using these sources. Specific ADRs, listed according to ease of study using these databases, are given in Table 2.Table 2: Adverse events according to ease of study using automated databases (aspects leading to ease or difficulty).Ad hoc cohort studies For epidemiologists, a cohort is simply a group of people followed over time during which health events are observed. In this sense of the term, the automated databases of the previous section can be used to form epidemiologic cohorts. The usable databases are circumscribed, however, by the kinds of data that are routinely captured. Ad hoc cohort studies, in which the data collection is specified in advance and implemented according to standard procedures, an in detection of ADRs when the and the diagnostic go beyond the occurs in automated The follow-up time of cohorts often that of clinical trials the of term and toxicities. data collection is ADRs such as and mood disorders can be may also be useful for of specific patient such as and drug – groups that are often from clinical trials and in automated A partial list of cohorts to HIV treatment is given in Table Note that the and duration of patient populations, data and number of list of cohorts available for study of adverse events associated with testing After a signal is through pharmacovigilance further is by way of hypothesis testing and formal studies. In particular, it is to both and quantify risk. is critical to to or the of the risk is critical to the importance of the ADR relative to other ADRs and other of and Spontaneous reporting systems Spontaneous reporting systems offer for formal hypothesis The significant under-reporting and lack of adequate data on number of patients exposed to a drug – as in the section – not ascertainment of the numerator (the number of people with an the thus making the of of AEs impossible. They are a valuable source of information between drugs or drug only in the of events that appear commonly in association with one drug or and almost in their Automated databases Automated databases have a potentially useful role in hypothesis Once an ADR is by spontaneous can an automated database for cases, and in order to for the use of automated databases for studies of ADRs in HIV requires several although database populations may be in the only a small of patients will have HIV, sample and the to detect only prescription information is – data on and are this information is based on prescription – it is not known if the was most automated databases are based on information – if a patient or they may be from the database. information on such as and adherence patterns are often not This limitation may result in the of significant regarding patient for treatment, and ascertainment is usually from care as only databases are to vital available source for this information is the but the has a time of approximately Ad hoc cohort studies Once an AE is large, patient cohorts can be used to associations of AEs with drugs or drug A major of cohorts over automated databases is that specific treatment and information can be more ascertained by than can be from medical For some HIV such as drug and observational cohort studies may be the only way to numbers of patients to safety Furthermore, the longitudinal nature of the data in many cohorts is particularly useful in the of duration of treatment on development of toxicity. Some such as the and the HIV also provide valuable and control Furthermore, to toxicity may be and – these data can in into the of major with the study of HIV drug toxicity, even in hoc is the lack of definitions for various ADRs This is particularly true for certain ADRs, such as lipodystrophy syndrome [11]. Furthermore, of case definitions the ability of cohorts to increase by Although individual cohorts may be large, for such as the number of case patients may This potentially useful has been to example of such is the study – a observational cohort study the association of with It involves cohorts patients who are followed for at least This study the way can increase sample and to detect infrequent but ADRs that otherwise be or to Another critical when cohorts as a source of toxicity data is that cohort studies are more to be by and than are clinical trials, where helps exposed and groups more Furthermore, issues of and the of may the ability of to identify less but significant All of these increase the time and for data which to and need for from both and Randomized clinical trials for the purposes of associations between antiretroviral drugs and long-term toxicities, RCTs are the However, currently RCTs have a on safety than on efficacy and this is true for both the and post-approval This is that the for specific of AEs could be by the control of known and unknown by the of multiple treatments and common in HIV, the of to two (or groups valuable insights into safety. example of a clinical to patients for an extended of time is or the This study to the long-term clinical of different strategies of antiretroviral therapy, and up beyond Other include the or and with 3 of and the Trials Randomized which will to ways to efficacy and toxicity of and will patients for These studies the and resource nature of the for these the use of HIV medications – from reporting to risk management In order to improve the use of HIV treatments to be by all groups involved to information on safety a major of HIV This result in a of to and scientific to the study of HIV-related AEs to this goal to into efforts specifically to improve all of drug including AE detection, and risk management. an that safety outcomes to their is to this is an understanding that in the of drug safety can in of critical is an understanding of the limitations of current knowledge of antiretroviral toxicities – is known and is This can be as a drug safety as shown in Figure derived from and The goal of pharmacovigilance in HIV be to the of the small as as The This to the most to the and study of ADRs – duration of follow-up (the and of the event (the The small in the upper the frequent in which more study and risk management of ADRs The the of the the unknown of those ADRs that occur and less post-marketing reporting of potential ADRs through spontaneous reporting systems is for enhanced understanding of long-term toxicities in pattern through is one way reporting of AEs may be In order to on ADRs be in both medical and as well as via for AE reporting Given a greater understanding of pattern recognition, will providers report more way to increase the of this may be to develop a specifically at AE This all of the from patient to Furthermore, an active the of ways to improve reporting be A limited example of this active in the UK, where spontaneous reporting is based on the This system an for clinicians to report suspected ADRs to the UK from this system that the of into the pharmacovigilance the number of reports received, which were of to those from In have been included as of AEs in the UK Furthermore, a system of the HIV drugs is an example of at In other as in where reporting of AEs has been for and regulatory steps have been to increase the of will need to be A but often of the surveillance is the Indeed, several long-term toxicities, including were initially recognized by HIV-infected individuals. Although MedWatch currently reports from some data that and patient may further pharmacovigilance For example, in patients currently are not to report a patients to suspected ADRs using reporting is The reports are communicated with a Pharmacovigilance which and the reports for further data that the system significantly increases the number of reports received, especially toxicities not or in Moreover, the system how such systems can be into surveillance systems. The be extended to include groups of providers, such as and The and under a standard process for data. Furthermore, the could be to if a particular AE signal found at one was also in are also in for AE signals from the of background 'noise' contained in the spontaneous reporting database. In this is in data a used to detect signals using data or AE background The data currently used by the signal for and (e.g. of drugs and events that are significantly more frequent than their associations and the of for the associations that can appear more when the number of events is small Given the of multiple drug and common in HIV the may have significant in the initial detection of ADRs related to the of HIV pharmacovigilance also involves AE reporting from cohort studies and clinical trials. most integral to this process is the of definitions of all known ADRs, of Furthermore, regulatory be on active surveillance through standardized of not only acute AEs but also on potential long-term toxicities such as mood and be and the of these events be included in results in a standardized In order to detect low-grade ADRs in clinical trials, it will often be to increase the sample the duration of follow-up or critical is the of patient groups in the study populations as studies are and for or and not only time and for AE reporting but require it for studies that otherwise be primarily on drug of the in this of also comprise an to pharmacoepidemiology in The of most clinical trials to and efficacy by not for in antiretroviral a ability to on study drug of some (e.g. for or ongoing ADRs that not lead to in therapy This can be by of safety data as to a that and for the number of those at risk as the study duration In ongoing cohort may have a on the of ADRs as time of ADRs be reported by the number of events by time of exposure. for time not account for other such as of time on drug and and the of (i.e. causing in a study of of these may require of data using such as a major of the of ADRs is their on of drug to ADRs with adherence be be on the of ADRs on of as this association to has been largely given the sample for detection and of more infrequent ADRs, further cohorts is In some cases, of association (i.e. for certain ADRs may be and by formal in this way increasing This of will not only require to and some of scientific but will also require be to support this of An of on ways to improve the of ADRs, specifically clinical trials may be found in a by Health care in the are referred for a more treatment of the Risk management Once ADRs are the communication of this information to not only health care providers but also Regulatory may on the information in a of including and Several of by the FDA on specific are listed in Table The efficacy of these however, has not been well studies include of the clinical and or process on of US Food and Drug Administration's regulatory in to adverse event reports for an is that of risk may only be defined over time, and therefore some regarding risk management have to be made prior to the understanding of the Thus, expectations that all safety be and risk management strategies be at their are However, the goal be to provide and providers with as information as is available on the and of HIV therapies as well as of the in that This goal necessitates that risk from be realistic and easily The potential role of the in data at the critical of patient when drugs are be and developed the that between drug efficacy and toxicity, between drugs and drug be subject to formal is a process which involves using the available evidence to a that health outcomes associated with under In of safety and the use of this process involves of ADRs, those a and the effects on various outcomes (e.g. treatment change of therapy, clinical This clinical limitations of the available the of certain and may result in better use of this will into clinical by more at the critical of patient Although it is recognized that certain patient and be easily included in these their results may in choice after treatment (e.g. the to have been monitoring of long-term toxicities associated with HIV an of that is in need of Indeed, to use medications we need to understand more the of toxicity and the of these toxicities on clinical this the of current therapies for a substantial number of be to be This report is based on and during a of of HIV by the for HIV The an international group of US and European drug HIV clinical HIV care providers, pharmaceutical and patient The is a which receives support from and as well as support from the patient and The was by the with from the of support from all of the and the and for their in with to the the management from and of the were for making the become a

BackgroundThe United States (US) Food and Drug Administration (FDA) approves new drugs based on sponsor-submitted clinical trials. The publication status of these trials in the medical literature and factors associated with publication have not been evaluated. We sought to determine the proportion of trials submitted to the FDA in support of newly approved drugs that are published in biomedical journals that a typical clinician, consumer, or policy maker living in the US would reasonably search.Methods and FindingsWe conducted a cohort study of trials supporting new drugs approved between 1998 and 2000, as described in FDA medical and statistical review documents and the FDA approved drug label. We determined publication status and time from approval to full publication in the medical literature at 2 and 5 y by searching PubMed and other databases through 01 August 2006. We then evaluated trial characteristics associated with publication. We identified 909 trials supporting 90 approved drugs in the FDA reviews, of which 43% (394/909) were published. Among the subset of trials described in the FDA-approved drug label and classified as “pivotal trials” for our analysis, 76% (257/340) were published. In multivariable logistic regression for all trials 5 y postapproval, likelihood of publication correlated with statistically significant results (odds ratio [OR] 3.03, 95% confidence interval [CI] 1.78–5.17); larger sample sizes (OR 1.33 per 2-fold increase in sample size, 95% CI 1.17–1.52); and pivotal status (OR 5.31, 95% CI 3.30–8.55). In multivariable logistic regression for only the pivotal trials 5 y postapproval, likelihood of publication correlated with statistically significant results (OR 2.96, 95% CI 1.24–7.06) and larger sample sizes (OR 1.47 per 2-fold increase in sample size, 95% CI 1.15–1.88). Statistically significant results and larger sample sizes were also predictive of publication at 2 y postapproval and in multivariable Cox proportional models for all trials and the subset of pivotal trials.ConclusionsOver half of all supporting trials for FDA-approved drugs remained unpublished ≥ 5 y after approval. Pivotal trials and trials with statistically significant results and larger sample sizes are more likely to be published. Selective reporting of trial results exists for commonly marketed drugs. Our data provide a baseline for evaluating publication bias as the new FDA Amendments Act comes into force mandating basic results reporting of clinical trials.

The self-fulfilling prophecy of post-hoc power calculations

Questionnaire surveys.

Do You Have Power? Considering Type II Error in Medical Education.

The Internet provides a convenient environment for data collection in psychology. Modern Web programming languages, such as JavaScript or Flash (ActionScript), facilitate complex experiments without the necessity of experimenter presence. Yet there is always a question of how much noise is added due to the differences between the setups used by participants and whether it is compensated for by increased ecological validity and larger sample sizes. This is especially a problem for experiments that measure response times (RTs), because they are more sensitive (and hence more susceptible to noise) than, for example, choices per se. We used a simple visual search task with different set sizes to compare laboratory performance with Web performance. The results suggest that although the locations (means) of RT distributions are different, other distribution parameters are not. Furthermore, the effect of experiment setting does not depend on set size, suggesting that task difficulty is not important in the choice of a data collection method. We also collected an additional online sample to investigate the effects of hardware and software diversity on the accuracy of RT data. We found that the high diversity of browsers, operating systems, and CPU performance may have a detrimental effect, though it can partly be compensated for by increased sample sizes and trial numbers. In sum, the findings show that Web-based experiments are an acceptable source of RT data, comparable to a common keyboard-based setup in the laboratory.

Conversion of Shoppers in Brick-and-Mortar Retailing - An Analysis of Observational Data

Genome-Wide Association Studies: Does Only Size Matter?

Increased availability of scanner-based panel data has enabled researchers to better understand nondurable commodity purchase dynamics. In this study, we focus on one component of the purchase process--when to buy. The relationship between the discrete purchase decision and a set of household and purchase characteristics is quantified using a simulated maximum-likelihood procedure. Given the longitudinal nature of our data, unobserved heterogeneity is addressed by adopting an auto-correlated error structure. Our empirical application is household purchases of cheese. We find evidence of significant persistent unobservable household heterogeneity, which is not eliminated by the inclusion of lagged exogenous variables.

This study investigates the utilization of Project-Based Learning (PBL) in SMA ICB Cinta Niaga to enhance students' business skills. PBL emphasizes applying knowledge in real-life situations, starting from concrete problems or projects. The method effectively boosts students' motivation by actively engaging them in learning. The research employs a qualitative descriptive approach with interviews as the primary data collection method. The study aims to evaluate the effectiveness of PBL in improving students' skills. Interviews with teachers and students at SMK ICB Cinta Niaga reveal the positive impact of PBL. Teachers acknowledge its role in developing business skills, while students experience significant benefits in honing their abilities. PBL creates a collaborative learning environment with challenges that encourage students to be active and responsive to problems. The study concludes that PBL at SMK ICB Cinta Niaga positively contributes to developing students' business skills. The method not only enhances learning motivation but also sharpens students' social skills in problem-solving. These conclusions are drawn from teacher experiences and student perceptions, portraying PBL as an effective and beneficial learning method for developing business skills in this educational context.

The aim of this paper is to analyse the impact of unobserved preference heterogeneity in empirical applications of discrete choice models of labour supply. Typically, unobserved heterogeneity is estimated either with continuous or discrete mixture models. However, in order to avoid estimation difficulties, most of the empirical analysis assumes a relatively constrained mixture, standard examples being models where only few coefficients are allowed to vary with independent normal distributions or with discrete distributions with few mass points. We compare labour supply elasticities obtained with these typical specifications of unobserved heterogeneity with those from a more general model that we are able to estimate through an EM algorithm for the nonparametric estimation of mixed models. Results show that labour supply elasticities change significantly with respect to a basic model without unobserved heterogeneity only when the joint distribution of the varying tastes is left completely unspecified.

The field of psychological science is in a pandemonium. With failures to replicate well-established effects, evidence for a skewed picture of science in the published literature, and a media hype about the replication crisis – what is left for us to believe in these days? Luckily, researchers have done what they do best – research – to try to establish the causes, and possible solutions to this replication crisis. A coherent picture has emerged. There are three key factors that seem to have led to the replication crisis: (1) Underpowered studies, (2) publication bias, and (3) questionable research practices. Studies in psychology often test a small number of participants. As effects tend to be small and measures noisy, larger samples are required to reliably detect an effect. An underpowered study, trying to find a small effect with a small sample sizes, runs a high probability of not finding an effect, even if it is real (Button et al., 2013; Cohen, 1962; Gelman & Weakliem, 2009). By itself, this would not be a problem, because a series of underpowered studies can be, in principle, combined in a meta-analysis to provide a more precise effect size estimate. However, there is also publication bias, as journals tend to prefer publishing articles which show positive results. Authors often do not even bother trying to submit papers with non-significant results, leading to a file-drawer problem (Rosenthal, 1979). As the majority of research papers are underpowered, the studies that do show a significant effect capture the outliers of a normal distribution around a true effect size (Ioannidis, 2005; Schmidt, 1992, 1996). This creates a biased literature: even if an effect is small or non-existent, a number of published studies can provide apparently consistent evidence for a large effect size. The problems of low power and publication bias are further exacerbated by questionable research practices, where researchers – often unaware that they are doing something wrong – use little tricks to get their effects above a significance threshold, such as removing outliers until the threshold is reached, or including post-hoc covariates (John, Loewenstein, & Prelec, 2012; Simmons, Nelson, & Simonsohn, 2011). As a lot of research and discussion exists on how to fix the problem of publication bias and questionable research practices – which mostly require a top-down change of the incentive structure. Here, I focus on the issue of underpowered studies, as this can be addressed by individual researchers. Increasing power is in everyone’s best interests: It strengthens science, but it also gives the researcher a better chance to provide a meaningful answer to their question of interest. On the surface, the solution to the problem of underpowered studies is very simple: we just have to run bigger studies. The simplicity is probably why this issue is not discussed very much. However, the solution is only simple if you have the resources to increase your sample sizes. Running participants takes time and money. Therefore, this simple solution poses another problem: the possibility of creating a Matthew effect, where the rich get richer by producing large quantities of high-quality research, while researchers with fewer resources can produce either very few good studies, or numerous underpowered experiments for which they will get little recognition*. On the surface, the key to avoiding the Matthew effect is also simple: if the rich collaborate with the poor, even researchers with few resources can produce high-powered studies. However, in practice, there are few perceived incentives for the rich to reach out to the poor. There are also practical obstacles for the poor in approaching the rich. These issues can be addressed, and it takes very little effort from an average researcher to do so. Below, I describe why it is important to promote collaborations in order to improve replicability in social sciences, and how this could be achieved. Why? In order to ensure the feasibility of creating a large-scale collaboration network, it would be necessary to promote the incentives for reaching out to the poor. Collecting data for someone with fewer resources may seem like charity. However, I argue that it is a win-win situation. Receivers are likely to reciprocate. If they cannot collect a large amount of data for you, perhaps they can help you in other ways. For example, they could provide advice on a project with which you got stuck and which you had abandoned years ago; they could score that data that you never got around to having a look at; or simply discuss new ideas, which could give you a fresh insight into your topic. If they collect even a small amount of data, this could improve a dataset. In the case of international collaborations, you would be able to recruit a culturally diverse sample. This would ensure that our view of psychological processes is generalisable beyond a specific population (Henrich, Heine, & Norenzayan, 2010). How? There are numerous ways in which researchers can reach out to each other. Perhaps one could create an online platform for this purpose. Here, anyone can write an entry for their study, which can be at any stage: it could be just an idea, or a quasi-finished project which just needs some additional analyses or tweaks before publication. Anyone can browse a list of proposed projects by topic, and contact the author if they find something interesting. The two researchers can then discuss further arrangements: whether together, they can execute this project, whether the input of the latter will be sufficient for co-authorship, or whether the former will be able to reciprocate by helping out with another project. Similarly, if someone is conducting a large-scale study, and if they have time to spare in the experimental sessions, they could announce this in a complementary forum. They would provide a brief description of their participants, and offer to attach another task or two for anyone interested in studying this population. To reach a wider audience, we could rely on social media. Perhaps a hashtag could be used on twitter. Perhaps #LOOC (“LOOking for Collaborator”)**? One could tweet: “Testing 100 children, 6-10 yo. Could include another task up to 15 minutes. #LOOC”. Or: “Need more participants for a study on statistical learning and dyslexia. #LOOC”, and attach a screen shot or link with more information. In summary, increasing sample sizes would break one of the three pillars of the replication crisis: large studies are more informative than underpowered studies, as they lead to less noisy and more precise effect size estimates. This can be achieved through collaboration, though only if researchers with resources are prepared to take on some amount of additional work by offering to help others out. While this may be perceived as a sacrifice, in the long run it should be beneficial for all parties. It will both become easier to diversify one’s sample, and help researchers who study small, specific populations (e.g., a rare disorder), to collaborate with others to recruit enough participants to draw meaningful conclusions. It will provide a possibility to connect with researchers from all over the world with similar interests and possibly complementary expertise. And in addition, it will lead to an average increase in sample sizes, and reported effects which can be replicated across labs. References Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafo, M. R. (2013). Confidence and precision increase with high statistical power. Nature Reviews Neuroscience, 14(8). doi:10.1038/nrn3475-c4 Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65(3), 145-153. Gelman, A., & Weakliem, D. (2009). Of beauty, sex and power: Too little attention has been paid to the statistical challenges in estimating small effects. American Scientist, 97(4), 310-316. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61-83. Ioannidis, J. P. A. (2005). Why most published research findings are false. Plos Medicine, 2(8), 696-701. doi:10.1371/journal.pmed.0020124 John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 0956797611430953. Rosenthal, R. (1979). The "File Drawer Problem" and Tolerance for Null Results. Psychological Bulletin, 86(3), 638-641. Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47(10), 1173. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1(2), 115-129. doi:10.1037//1082-989x.1.2.115 Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 0956797611417632. Footnotes * One may or may not consider this a problem – after all, the issue of the replicability crisis is solved. ** Urban dictionary tells me that “looc” means “Lame. Stupid. Wack. The opposite of cool. (Pronounced the same as Luke.)”

We compare farm level efficiency rankings derived from non-spatial and a variety of spatial model specifications that account for unobserved heterogeneity in both the production and the efficiency sides of the stochastic frontier model in an empirical application on rice farming in the Philippines. We show how not accounting for unobserved spatial heterogeneity affects efficiency estimates and farm efficiency rankings. When not accounting for unobserved spatial heterogeneity efficiency, models show farms to be relatively more inefficient than they actually are (i.e., once unobserved spatial heterogeneity is incorporated in the models). More importantly from a policy perspective, the rankings of the farms in terms of efficiency are altered once unobserved spatial heterogeneity is incorporated in efficiency models. We recommend the use of unobserved effects in both production and efficiency within the stochastic frontier analysis framework to avoid making any misleading recommendations to farmers and policymakers.