I think I figured out my issue. When using a TTF with Fill set, no matter what color/pen your text is, it does not get drawn with any pen thickness (similar to how pen 16 works, minus drawing in color). I can turn on outline and then the outline will adjust itself using the color/pen settings. Unfortunately right now I have all of our standards built using just fill and I don't want to go back and add a outline as the color pen that I would have to use would be different that the fill color and then when working in the drawing it looks like the text is the wrong color until you zoom in enough to see the fill color.
Thanks! - Josh Do. Or do not. There is no try. Josh's Digital Downloads is come back online soon. Stay tuned. (soon is a relative term)