Not illegal IMO and the people behind you wouldn't like it (and potentially unsafe since most people following aren't expecting it). It appears the most relevant VC sections would be 22102 and 21651(a)(2) in this case. If there were a sign prohibiting the turn/u-turn, that would obviously be a different story. As with some laws, interpretation can differ from one person to the next.
22102.
No person in a business district shall make a U-turn, except at an intersection, or on a divided highway where an opening has been provided in accordance with Section 21651. This turning movement shall be made as close as practicable to the extreme left-hand edge of the lanes moving in the driver’s direction of travel immediately prior to the initiation of the turning movement, when more than one lane in the direction of travel is present.
21651.
(a) Whenever a highway has been divided into two or more roadways by means of intermittent barriers or by means of a dividing section of not less than two feet in width, either unpaved or delineated by curbs, double-parallel lines, or other markings on the roadway, it is unlawful to do either of the following:
(1) To drive any vehicle over, upon, or across the dividing section.
(2) To make any left, semicircular, or U-turn with the vehicle on the divided highway, except through an opening in the barrier designated and intended by public authorities for the use of vehicles or through a plainly marked opening in the dividing section.
(b) It is unlawful to drive any vehicle upon a highway, except to the right of an intermittent barrier or a dividing section which separates two or more opposing lanes of traffic. Except as otherwise provided in subdivision (c), a violation of this subdivision is a misdemeanor.
(c) Any willful violation of subdivision (b) which results in injury to, or death of, a person shall be punished by imprisonment pursuant to subdivision (h) of Section 1170 of the Penal Code, or imprisonment in a county jail for a period of not more than six months.
DMV Handbook regarding this topic:
"In business districts. Areas with churches, apartments, multifamily housing units, and public buildings (except schools) are also considered to be business districts. Turn only at an intersection, unless a sign prohibits it, or where openings are provided for turns."