Installing Your Headlamp Strap
The Easy Way and the Not as Easy way
There are two basic ways that your replacement strap may attached to your headlamp.
The first way, the headlamp strap is slipped on through splits in the loops that hold the strap. this would look something line this:

If that is the case , life is easy.. slip your old one out via the splits, slip the new one in and you are ready to go.
The other option for attachment is the strap being threaded through closed loops such as pictured below. 
There is a little more work involved getting your new strap on, but the instructions below should get your through easy peasy. Remove you old headlamp strap first. The fast way is to cut it off with scissors, but if you want to save the old one for whatever reason, it will probably come off by reversing the installation instructions, so just start with the last step below and go backwards.
Strap Installation for Closed Loop Headlamps
First
Pull the strap away from the the Split Loop (the larger plastic hardware)

Second
Remove the strap end from the Split Loop.. note there is a split to slip it out of.

Third
Thread the strap through the split loop the same way, so the Split Loop is free.

Forth
Backout the the loose end through the Single Loop (the smaller plastic hardware piece).

Fifth
Thread the end through the single first loop of your headlamp. For neatest appearance the headlamp should be on the outside of the strap, the inside being the where the cut ends of the strap are visible.

Sixth
Thread the end through the other side of the headlamp and pull strap through so there is a plenty of length to work with.

Seventh
Thread the working end back through the Single Loop.

Eight
Slip the Split Loop back onto the strap as pictured.

Nineth
Slip the working end into the split of the Split Loop and adjust so that the working end is held in place by both sides of the spilt.

Tenth (and finally)
Adjust Your strap for the perfect fit.

There is a Wrong Way
There is a wrong way to put together the strap, as pictured below.
The top strap is correct, where the single loop hardware has the strap coming from the split loop and returning to it. The bottom strap is incorrect , as the single loop can slide back and forth along the strap. This can be fixed. If you strap is incorrect , remove the split loop (as in steps 1 -3 above) , and then swing the loose end to the other side and reattach the split loop as in steps 8-10 above)