My sister's husband harassed me!

Hi, I’m a 34 year old women and I do a job. Actually I have a love marriage with my husband and its an intercaste marriage, its been 1 year that we are married. I have one sister in law and she has been married from past 7 years, her husband is a very rich and weird. From beginning he used to see me or touch me very awkwardly but I thought its me who is getting wrong thoughts. In one of our family gathering he was making jokes and having fun. But later he physically harassed. He touched and did something’s very inappropriately. I went to my husband and tried to express what happened to me. He acted cool and said will go home and discuss. He went back to enjoy the party. I felt bad. We went home after 1 hour. He ignored the issue, next day when he wokeup, I told him again that I felt bad. He said forget about it, he must be drunk and he is my brother-in-law. So nothing happened. I raised this concern to my family and they came to discuss at the end big issue happened. I stopped meeting his sister and sister’s husband. This issue never got solved & we ended up divorcing. Now I feel that I made a big issue out of a small one and feel guilty about it. Did I do right or wrong.