Treatment of water at the household level offers a promising approach to combat the global burden of diarrheal diseases. In particular, chlorination of drinking water has been a widely promoted strategy due to persistence of residual chlorine after initial treatment. However, the degree to which chlorination can reduce microbial levels in a controlled setting (efficacy) or in a household setting (effectiveness) can vary as a function of chlorine characteristics, source water characteristics, and household conditions. To gain more understanding of these factors, we carried out an observational study within households in rural communities of northern coastal Ecuador. We found that the efficacy of chlorine treatment under controlled conditions was significantly better than its household effectiveness when evaluated both by ability to meet microbiological safety standards and by log reductions. Water treated with chlorine achieved levels of microbial contamination considered safe for human consumption after 24h of storage in the household only 39-51% of the time, depending on chlorine treatment regimen. Chlorine treatment would not be considered protective against diarrheal disease according to WHO log reduction standards. Factors that explain the observed compromised effectiveness include: source water turbidity, source water baseline contamination levels, and in-home contamination. Water in 38% of the households that had low turbidity source water (<10 NTU) met the safe water standard as compared with only 17% of the households that had high turbidity source water (>10 NTU). A 10MPN/100mL increase in baseline Escherichia coli levels was associated with a 2.2% increase in failure to meet the E. coli standard. Higher mean microbial contamination levels were seen in 54% of household samples in comparison to their matched controls, which is likely the result of in-home contamination during storage. Container characteristics (size of the container mouth) did not influence chlorine effectiveness. We found no significant differences between chlorine treatment regimens in ability to meet the safe water standards or in overall log reductions, although chlorine dosage did modify the effect of source conditions. These results underscore the importance of measuring both source water and household conditions to determine appropriate chlorine levels, as well as to evaluate the appropriateness of chlorine treatment and other point-of-use water quality improvement interventions.