The lattice dynamics of CsSnX3 (X = Cl, Br, and I) and CsPbI3, which are low-thermal-conductivity materials, are investigated using first-principles phonon calculations. Because of the strong lattice anharmonicity and the accompanying instability of high-temperature cubic phases, the self-consistent phonon theory, which can incorporate the effect of lattice anharmonicity at a mean-field level, is applied in this study. The calculated lattice thermal conductivity reproduced a low thermal conductivity, as shown experimentally, owing to the short phonon lifetime due to the incoherent scattering contribution of Cs atoms. The halogen ion dependence on thermal conductivity reveals that CsSnCl3 exhibits an anomalous lattice thermal conductivity that is as low as that of CsSnBr3. This indicates that the lattice dynamics cannot be explained merely in terms of the atomic mass of the compounds. The low thermal conductivity of CsSnCl3 is caused by the exceptionally short phonon lifetime; further, a bonding analysis suggests that covalent bonding contributes significantly to the unusual anharmonicity of CsSnCl3.