Find a homogeneous linear system of two equations in three unknowns whose solution space consists of those vectors in r3 that are orthogonal to a=(1,1,1) and b=(-2,3,0)

Basically, you're looking for a matrix such that



i.e. a matrix whose nullspace with basis vector .

By the rank-nullity theorem, the rank of and the dimension of its nullspace must add up to the number of columns, so



One easy choice for a row would be , since



Now you only need to find another combination such that the second row of is independent of the first. An easy choice for this is to let the first element be 0, and the next be 1. Then the last element must be , as



So,



is one possible solution.