In this study we propose a framework to reconstruct 3D pose of a human for animation from a sequence of single view video frames. The framework for pose construction starts with background estimation and the performer's silhouette is extracted by using image subtraction for each frame. Then the body silhouettes are automatically labeled by using a model-based approach. Finally, the 3D pose is constructed from the labeled human silhouette by assuming orthographic projection. Proposed approach does not require camera calibration and it assumes that the input video has a static background and it has no significant perspective effects and the performer is in upright position.