Could you please suggest algorithms for solving non-convex integer optimization with constraints? The search space is very large, so branch and bound does not seem practical. A few methods I have tried:
Simulated annealing Genetic algorithm Differential evolution
First, there are solvers designed specifically for non-convex discrete optimization. I'm not personally familiar with them, but BARON comes to mind.
Second, there are insanely many metaheuristics to choose from. Not all are applicable to problems with constraints and discrete variables, but even with those requirements there are still quite a few. You might look at the Evolutionary Computation Bestiary for papers on metaheuristics that fall into the broad category of evolutionary algorithms (like genetic algorithms). Frankly, I have no idea how to choose among them other than by trial and error. Each has its proponents (some of whom border on zealots) and each has its detractors. Not that I've ever used it, but among the evolutionary algorithms my favorite is "slime mold optimization" just for its name.