There are two ways to choose an agency.
Choose an agency that you might like to work for or addresses a public policy problem that your are interested such as cleaning up the environment, ending welfare as we know it, fire depts, monitoring elections, giving out drivers' licenses, inspecting the cleanliness of restaurants, preventing drug use or cigarette smoking, state parks, the Adirondacks, the Statue of Liberty, highways, prisons, regulating casinos/Indian gambling), and then figure out what agency is responsible for solving the problem. See below for where to find a list of agencies.
My preferred method. Browse through the phone book or state and local government web pages to find an agency that looks interesting. The New York State Statistical Yearbook in the library has an Appendix with a directory of major New York State Government Agencies and other sources with brief descriptions of what the different agencies do. You can also look at the official state web page which has links to all sorts of good stuff, especially the link to all federal, state, and local government agencies. You can also look at this link to the Empire Page politics web page about politics in NY, especially the resource links at the bottom. You can also look at the Saratoga County web page or the City of Saratoga Springs web page.