It sounds like you need more brake (sometimes called reverse), which you've kind of figured out on your own. When you slid your hands down the handle you added more brake. The fluttering of the sail is usually due to not having enough brake.
If your handles have knots on the attachment lines, attach the top lines from your kite to a knot further out from the handles.
If you don't have knots, it's a good idea to add them. Someone here will post a link, I don't have one handy. If someone doesn't, I'll track one down later.