I would charge more. Think of it this way: with wholesale, you are getting your brand spread farther and you're reaching customers you wouldn't have otherwise. With the private label, you're doing the same amount of work and not reaping the benefit of growing your brand. Rather, the other company is using your labor to grow *their* brand. You're providing a product and a service.
Only you know your business expenses and revenue goals so I don't know if anyone can truly tell you the exact rate you should charge, but in my opinion it should definitely exceed your wholesale prices.