Task Reminder emails can be styled to your individual liking with Cascading Stylesheets (CSS). Your CSS code will be directly applied to the HTML/Velocity code in users' email clients.
Consider the following simple dummy content rendered with the default template:
Here's a stylesheet that will change the appearance to the following (I'm not intending to win a prize in style here...):
This is the stylesheet for the above appearance: