One essential problem in precision medicine is to determine optimal Individualized Treatment Rules (ITRs) that tailor treatment decisions to patient-specific characteristics so as to maximize rewarding outcome. In practice, many treatment options are usually available, and this poses a significant challenge for learning reliable treatment rules given limited data. In this work, we propose GRoup Outcome Weighted Learning (GROWL) to simultaneously learn both group structure among the treatments and the resulting optimal group-based ITRs. Our approach combines treatment clustering and value optimization through one single algorithm. We provide theoretical guarantee for GROWL by establishing the results for Fisher consistency, excess risk bound, and non-asymptotic convergence rate. Extensive simulation studies and real data analysis are performed to demonstrate the superior performance of this method.