When you read a post, at the bottom is little information about the author.  There can be a picture and a biographical section.  This also will show up when someone comments on the post.  This article explains how you can fill in that information.

There are two ways for your picture to show up.

Here’s the first way

After logging in you should go to your profile page. This is where you can change your name, email, password etc. At the bottom is a place to upload a picture. Click “upload” and browse your computer for a good picture. You can change the picture anytime you like so don’t worry too much.
And while you are there, go ahead and fill out your bio information so everybody knows a little about your. So do it now, just click on the website section “Your Profile” and look it over.

Here is a picture of what that section should look like.

Once you upload it, it will show up when you comment on a post, or if you are an author. That should look a little like this.

The Second Way to display a picture.

WordPress is a blog among hundreds of blogs and as such, created a way for the same picture of you to show up no matter what website you may have posted on.   To facillate this, a website was created called http://en.gravatar.com/ And this is the site you need to go to and create your own avatar.  The site explains what an avatar is.  Basically just register using the same email address you used here, and upload a picture you want shown.  You can upload several, change them at anytime and it doesn’t even have to be a picture of you.  You can use your favorite flower, thing, animal etc. Once you do that, this site will find the picture automatically.

So just to recap, if this is the only place you might ever comment on a post, just go to your profile. If you comment at a lot of different websites, the gravatar is probably the better way to go.