Don't be scared, just don't go to certain areas of "town." Anything near or on the OBT is probably a good place to avoid. That is a strip that gets worse and worse with passing time. Same for anything downtown that is east of I-4 (I know I-4 runs east and west, but through downtown Orlando until above Lake Mary it runs north and south!) or Division Street. Those are the main areas where drug trafficking and prostitution are prevalent. There is quite a bit of panhandling downtown too, so be alert to that. Some panhandlers can become quite aggressive.