So, to first order, an infinitesimal rotation matrix is an orthogonal matrix.

But one must always be careful to distinguish (the first order treatment of) these infinitesimal rotation matrices from both finite rotation matrices and from Lie algebra elements.

When contrasting the behavior of finite rotation matrices in the BCH formula above with that of infinitesimal rotation matrices, where all the commutator terms will be second order infinitesimals one finds a bona fide vector space.