A novel method that may be used to interactively control the micro-environment for young swine was investigated to classify the thermal comfort state of animals. Early weaned pigs at 13 to 16 days of age were housed in groups of 10 pigs in four environmentally controlled chambers (1.52 m × 1.83 m floor space per chamber). Air temperatures inside the four chambers were set at 24.4°C, 26.7°C, 28.9°C, and 31.1°C, respectively, for the first week , and were reduced by 1.1°C each of the following two weeks. Postural behaviors of the pigs (huddling or spreading) were captured every 40 min with programmable cameras installed above the transparent false ceilings of the chambers. The raw behavioral images were processed by thresholding, edge detection, and morphological filtering techniques to separate the pigs (objects) from their background. The processed images were further subjected to Fourier transformation. The Fourier coefficients of the processed images (8×8 features) were then used as the inputs to a neural network, which classified the environment into cold, comfortable, or too warm category for the pigs. The neural network analysis worked quite well, with 131 out of 136 training images (96%) and 51 out of 65 testing images (78%) properly classified. This study demonstrates that an innovative environmental controller which uses the animal behavior, instead of the conventionally used air temperature, as the input variable, is possible for swine production. It is anticipated that the behavior-based automatic controller would lead to improved animal well-being and production efficiency. Future research needs include development of algorithms for automatic image segmentation of the pigs, exploration of alternative feature extraction methods to improve classification accuracy of the neural network, and development and evaluation of the behavior-based controller prototype.