Replace byte[] with color struct

[Shane32]

Instead of the byte[] could we use a

public struct Color
{
    public byte R;
    public byte G;
    public byte B;
    public byte A;
}

to avoid the (temp.) allocation?

Or when the array-like semantics should be kept, an InlineArray (where supported) / a fixed sized buffer when not supported?

Could be done in a separate PR also. Maybe that would be event better, as such "helper types" can be created there, and with a pass over the code base more such occurances be changed.

Originally posted by @gfoidl in #640 (comment)