Why did swine flu used to be such a big worry?

Because it's the first time a swine virus mingled with a human virus. Swine viruses have never crossed over into humans before and no one knew what to expect.