I believe you can use OptiGrating to advantage, but I’m not sure exactly what your grating looks like. Is it made by modulating the width of a rib waveguide? Or is it a slab waveguide with two surface gratings applied to different layers?

In either case, one can design these gratings with OptiGrating, but this application needs some optical parameters. Analysis of a grating by the coupled mode theory (CMT) is done in two parts. The first part is the reduction of the electromagnetic fields of the waveguide to parameters such as modal index and overlap integrals of the modes with the grating. The second part is the application of CMT to those parameters to calculate the reflection and transmission spectra.

OptiGrating comes prepared to model the first (electromagnetics) part of the problem for waveguides such as optical fibres and slab waveguides with a single surface grating. One can still use OptiGrating for more general waveguides, but you’ll have to find the relevant parameters yourself, maybe by means of another software, for example, OptiMode.

In a new OptiGrating project, select the option “Other Waveguide”. In the new project, the menu item for waveguide design is disabled, but the Mode item is still functioning. It is possible to enter the relevant parameters such as modal index and overlap integrals there. Once these are defined, grating design can proceed in the same way as fibre grating design.

One of the Optigrating samples, moiregrating.ifo, is an example of a grating defined as Other Waveguide.