Ankle weights generally aren't recommended for brisk walking.
Although ankle weights can increase the energy you burn while walking, they may strain the ankle joint and leg muscles, which could increase your risk of injury.
To get more out of your walking routine, simply try picking up the pace. If you're in good shape, add short bursts of jogging into your regular brisk walks. If you're less fit, alternate leisurely walking with periods of faster walking. For example, if you're walking outdoors, you could walk faster between certain mailboxes, trees or other landmarks.
If you'd like to include strength training in your fitness routine, you have plenty of options besides weights. For example, consider resistance tubing — these elastic-like tubes offer weight-like resistance when you pull on them. Your own body weight counts, too. Try pushups, pullups, abdominal crunches and leg squats.