We introduce the general optimal stochastic control setting in the case of portfolio selection. We implement an Optimal Control Projection Algorithm (OCPA) to solve the objective function while using projection, kernel estimation and solving for a solution in a single index context. In addition, we propose a lower bound of the value function which can be used to test the validity of the estimated sub-optimal. We compare the performance of OCPA and the popular EM algorithm using both the simulation and empirical study. In general, the OCPA is doable when the EM algorithm is not, and the OCPA is much faster and performs better when the EM algorithm is doable.