Poverty reduction in low income countries is seen as an important line of defense against the global HIV/AIDS pandemic. With proper livelihood diversification, households with asymptomatic HIV members can achieve greater sustainability and wellbeing. Little is known, however, about factors that affect diversification of livelihood activities among such households. This study defines `proper diversification' as inclusion of non-farm activities in households' livelihood portfolios. The non-farm activities in this study are represented by self-employment, working for wages, and remittances. This study employs analysis of variance (ANOVA) and structural equation modeling (SEM) methods to study the effects of selected individual and household level factors on `proper diversification' in the panel of 364 Ghanaian households with HIV positive and HIV negative mothers who recently gave birth to a child and was observed over the 12 month period following birth. Results indicate that households with HIV positive mothers have lower socioeconomic status, lower human capital, and HIV positive women experience higher levels of psychological stress. Moreover, these households experienced reduced `proper diversification' during the period of observation. Further, human capital expressed in years of education of heads of households is an important predictor of `proper diversification'. Its effects are even stronger in households with HIV positive women. Inconclusive results were noted regarding moderating and mediating effects of socioeconomic status, social capital and stress on households' `proper diversification.' This result should be partially attributed to the study sample which was based on a self-selection process. This limitation should be considered in the future studies. This study makes two main recommendations to policy makers and development agencies. First, new innovative income generating opportunities need to be offered to households affected by HIV to promote `proper diversification.' Second, methods to encourage females to remain in schools to achieve higher levels of education attainment can have positive impacts on livelihood diversification and enhance responses to the HIV pandemic.