Concerns about crime and anti-social behaviour have been spotlighted by worried locals in a town centre close to England's second city Birmingham.