Geometry: 2 lines are orthogonal if they meet at a right angle (90 deg). But the lines are independent.
Computing: 2 parts of a system are orthogonal if changes in one do not affect the other. Other words for orthogonal are modular, layered, component-based.
Benefits
orthogonal systems are easier to change (ETC)
encourages component-based development
gain productivity
changes pertain only to the one component -> less development or refactoring time
reuse of components is promoted into combinations of such components
reduce risk
bugs are isolated to single components (unless they are combined)
writing unit-tests for components is done easily
Orthogonality Tests
"If I change the requirements behind a particular function, how many modules are affected?" (The Pragmatic Programmer by D. Thomas and A. Hunt)
How decoupled is your design from real-world changes? (3rd party plugins come here to mind)
Do you write unit-tests easily or do you have to do a difficult setup for the tests with a large part of the system's code?
After fixing bugs, do problems in other parts of the code appear?
Techniques to Designing Orthogonal Systems
Keep your code decoupled.
Avoid global data.
Avoid similar functions. (e.g., Strategy Pattern)