It's unlikely to happen, I know, though it would be a good thing to do. But if legislators took the idea of abolishing the NSA as under discussion - even if they don't expect it to happen - it would shift public perception to make other, less drastic but still reasonable changes more likely.
You can find your Senators' and Representative's phone numbers at whoismyrepresentative.com, by putting in your ZIP code. Call their DC numbers, the ones with 202 area codes.
P.S. Why do I think asking to abolish it as not just a way to move the politics, but also a good idea? Having such a secretive organization that routinely disobeys laws and lies about it as long as they can get away with lying, because they expect not to get caught because of their secrecy, makes them nearly entirely outside of democratic government. As they grow bigger and more powerful, more and more our government is no longer under the authority of democracy. Other agencies, such as the FBI and CIA and military, show they're perfectly capable of investigating crimes, undercover work, collecting intelligence, and so on, while still working under democratic authority; something like the NSA is clearly not needed. While the NSA does some useful work, that work would be much more useful if moved to more appropriate agencies. For example, we'd get much greater public benefit from their cryptography work if it was moved to NIST and/or the NSF. Some of their functions - and people - could be moved into the CIA and FBI.