This is happened because filter feature filtering only visible portfolio/blog items and if you use load more with filter combination then you will get this issue, because by default you have for example 10 visible items and if you click on some category inside filter you will get only items from that set of items (in this case 10) inside that category and you will get that issue. So you first need to click on load more then to filter again your items and you will get other items from that category.
Unfortunately there is no any fix for that to filter load all items from that category, because script which we are using for the filter is working in that way :(
So, only solutions are:
a) to set all items to be visible without Load More functionality, to do that just edit yoour portfolio list shortcode and find Number option and file -1 instead of 8 and save
b) to disable filter option
c) to increase number of portfolio items on your page so you can have more items for each category