Conventional algorithms process addition and subtraction in different ways. In this paper, by introducing bipolar carries, a unified negabinary symbolic arithmetic for the two operations can be derived, using only the same six substitution rules. Based on the polarization-encoded optical shadow-casting technique, the encoding and decoding patterns for half-addition, half-subtraction and unified arithmetic are designed, with the fixed LED pattern and separate and simultaneous generation of the multi-outputs. The architecture is simple, programmable, cost-efficient, and the addition and subtraction of multiple pairs of bipolar numbers can be handled in parallel in the same manner.