Having a bad manager can ruin an otherwise great job. It makes a huge difference.
I was in the same situation when I left for maternity leave. I loved mat leave, but many people don’t. Yes, it was tough at times, but I overall loved it. I have friends who are also on mat leave and I met other new moms, so I had something to look forward to every day. Before I left work, I made it clear that if I had to return to my then-manager, I wouldn’t be coming back. They made a manager switch happen, so I planned to return.
Going back was so so hard. It’s been a week, and I cry every morning. I don’t generally cry much. But I’m planning to stick it out for at least a quarter to make sure I don’t change my mind about staying home. For what it’s worth, I actually enjoy my job. I think it’s meaningful, I’m paid enough to make it worth my time, it’s interesting enough, and I have decent autonomy. Overall, a decent job with good work-life balance. But I’m still thinking of quitting to stay home.
Long story short, no, you are not the only one.