On JUNOS routers, routing-instances are used to separate OSPF instances, and JUNOS follows RFC5838 to use ID's for different address families (AF's):
When turning on JUNOS protocols ospf3 traceoptions flag all, we can see the trace log indicating the issue:
Cisco router as our OSPF3 neighbor may configure like the following:
JUNOS is working as per RFC5838 standard. A neighbor router to talk OSPF3 for IPv6 family needs to configure instance ID comply with RFC5838.