This paper presents a model-free approach to automate folding of a deformable object with robot manipulators, where its surface was labelled with markers to facilitate vision-based control and alignment. While performing the task involves solving nonconvex or nonlinear terms, in this paper, linearization was first performed to approximate the problem. By using the Levenberg-Marquardt algorithm, the task of folding a deformable thin object can be reformulated as a convex optimization problem. The mapping relationship between the motions of markers on the image and the joint inputs of the robot manipulator was evaluated through a Jacobian matrix. To account for the uncertainty in the matrix due to the deformable object, a two-stage evaluation scheme, which consists of approximate-rigidity rule and Broyden-update rule, was performed. Proper constraints were also added to avoid causing damage to the object. The performance and the robustness of the proposed approach were examined through simulation using Bullet simulator. The video of the simulation can be retrieved from the attachment. The results confirm that the thin object can be precisely folded together based on different markers labelled on the surface.