Accurate 3D reconstruction of buildings has many applications including remodeling, renovation, or recladding. Existing software such as Revit and VectorWorks are primarily for designing new buildings, which by definition must be plumb, true, level, with 90 degree Manhattan geometry; as such they are not suitable for Scan-to-BIM applications where the building facades might not be vertical, flat, plumb, true, level or Manhattan. This is particularly true of old existing buildings in need of energy retrofit. In this paper, we develop a novel automatic pipeline for creating accurate 3D representation and modeling of a building beginning with a 3D point cloud. We use feature extraction and clustering methods in order to segment the original point cloud into its respective facades, roof, and ground, followed by the polygonization of each facade. Next we delineate the boundaries of fenestration such as doors and windows. We test our approach on two multifamily apartment buildings, two in Southern California and one in Canada. We characterize the accuracy of our method for one of the buildings in Southern California using hand measurements and tape, and show that our approach results in 33 to 50% less error than the corresponding Revit model.