The Challenges of Building Mixed Reality Games - 80 Level Podcast

The founder of Synodic Arc Michael Pulis talked about his experience working on mixed reality titles and the future of the metaverse. 

Check out Synodic Arc’s new Mixed Reality game Zombies Noir: https://www.oculus.com/experiences/quest/4745920782133500/

Visit Synodic Arc website: https://synodicarc.com/

Michael Pulis a Lead Programmer and Founder at Synodic Arc

Follow 80 LEVEL on social media:
https://www.facebook.com/LevelEighty
https://www.instagram.com/eighty_level/
https://twitter.com/80Level

We are looking for more artists!
Join 80 LEVEL Talent for free: https://80lv.pro/join-80lvTalent
Get your work noticed by some of the biggest and best developers, publishers, and studios in video games today.

The Gaming Blender
Could you design a video game?

Listen on: Apple Podcasts   Spotify

This video is sponsored by Xsolla, a global video game commerce company with a robust and powerful set of tools and services designed specifically for the video game industry: http://xsolla.pro/8023

Transcript

0:00

foreign

0:07

go into the nitty-gritty can you do like

0:09

a little introduction to us tell us a

0:12

bit about yourself your company I know

0:14

that you're a main lead programmer but

0:17

you're also a CEO so I imagine it's a

0:19

smaller company to tell us a little bit

0:21

about how it started sure so uh my name

0:24

is Michael polis I'm the founder and

0:26

lead programmer for sonatic Arc we're

0:29

based in Bellevue in Washington in the

0:32

United States so we're working on a

0:34

couple projects right now primarily uh

0:36

finishing up work on zombies Noir which

0:39

is a mixed reality game and that

0:42

launched actually last week in Early

0:44

Access on the Atkins Quest 2 app lab and

0:47

you know this chance will be important

0:49

as other Platforms in the future but

0:50

that's our primary target right now just

0:52

trying to get feedback out there about

0:54

the game

0:56

so

0:58

um I think this is an idea that has been

1:01

in the air for so long the mixed reality

1:04

thing and we've seen a couple of

1:07

examples and I still see a very

1:09

interesting examples of

1:11

how this can be

1:13

done in an entertaining way and how you

1:16

can build an entertaining product around

1:19

it so before we're gonna go into

1:21

thinking about it in a broad way can you

1:25

tell us a bit about how do you approach

1:27

this in your game and what can be done

1:30

in your title so I guess obviously the

1:35

first thing people think of are should

1:36

think of in mixed reality moving forward

1:38

is basically combining like the real

1:40

world with the virtual world so what we

1:44

do is uh with the I guess Quest 2's room

1:47

setup feature you're able to basically

1:50

construct your room with walls and map

1:53

out Furniture to basically allow you to

1:56

interact with those elements what we do

1:58

more specifically though is we let

2:00

players uh place door doorways in the

2:04

game itself on your walls in your room

2:06

so what that means is that your room is

2:09

essentially acting as a portal for

2:12

zombies to come from their world into

2:14

our world so what you'll see is once you

2:16

place these you know door portals on the

2:18

wall you'll see the zombie world out

2:20

there and then they're going to come in

2:22

from their world into our world and

2:24

that's all going to

2:26

um you know work with the icons Quest

2:28

2's occlusion system

2:31

um there's no like stretching or

2:33

anything it all feels very natural and

2:35

realistic which is very cool

2:37

so um we talked a bit about

2:41

how it works in your game and we have

2:45

seen a lot of examples of how it's done

2:47

differently like probably one of the

2:50

most prominent examples we've seen

2:52

recently was when a guy turned his whole

2:56

house into like a SWAT uh simulator

2:59

where he was just doing you know like a

3:01

ghost recon kind of style game and there

3:05

was like his sofa became the cover he

3:08

would you know Traverse his and shoot

3:10

like guys that were attacking him

3:14

um

3:15

from from what you see just on the

3:18

purely on the side of like when the

3:20

project is done it seems like it's uh

3:23

very easy to implement and my question

3:25

for you is

3:27

how difficult and challenging it is from

3:31

a technical standpoint to create uh like

3:33

a mixed reality title because there are

3:36

a lot of elements that you don't really

3:38

think about when you build your you know

3:41

like a virtual title where everything's

3:43

designed inside the engine and here you

3:46

have a lot of different other things

3:47

like what are the things that are very

3:49

different here I think more so than from

3:53

a technical perspective a heart one of

3:55

the hardest things is the design

3:56

perspective

3:58

um I guess maybe we can start with the

3:59

technical issues so it really depends

4:03

what you're planning to do with a mixed

4:04

reality title so what is nice is you

4:07

have so many things built in

4:09

automatically from the Oculus Quest 2's

4:12

you know room setup feature which just

4:14

helps you you know work in this mixed

4:16

reality space so that that part is

4:19

actually pretty simple it gets a lot

4:21

more difficult once you start needing to

4:23

have different uh I guess specifications

4:25

for every single type of thing you're

4:28

working in are working on so if you want

4:31

your walls to be different from your

4:32

ceiling different from your floor

4:33

different from your furniture all of

4:36

those things can be done fortunately but

4:38

it is difficult to make each of them

4:39

work correctly 100 of the time

4:42

um

4:43

and then on top of that if you're doing

4:45

anything where you're you know severely

4:48

changing the I guess what you would say

4:52

um like the art of uh stuff that can

4:55

also become a little bit challenging to

4:57

work with so

4:59

um you need to make sure that the size

5:00

specifications are mapped out correctly

5:02

which you could probably do with some

5:04

scaling

5:05

um but we have found in our experience

5:07

that sometimes things from the API don't

5:10

map out one to one to what you would

5:11

expect so it can take a little bit of

5:13

finagling to get that to work

5:16

um let me speak on the design side of

5:18

things however I think the hardest part

5:20

is finding just a way to make playing in

5:24

your house interesting

5:26

so with a normal VR title the first

5:29

thing that's going to come to mind is

5:30

you know you have this Infinite Space in

5:32

the game world and the game engine it's

5:34

really up to you as the the game

5:35

developers to make that space fun but

5:40

um as a mixed reality title we have to

5:43

make sure that every single player's

5:45

room is fun to play in it doesn't matter

5:47

how big or how small we need to make

5:49

sure that every size of room with every

5:52

set of furniture is going to be fun for

5:54

the player and I think that is one of

5:56

the biggest constraints

5:58

um in developing a mixed reality

5:59

experience that has caused us the most

6:02

trouble

6:03

we've seen some examples where mixed

6:08

reality is kind of linked to

6:10

mobile devices and that's kind of like

6:13

my uh

6:15

that's where we see a lot of interest

6:17

from investors and companies and you

6:20

know trying to blend in the real world

6:24

and

6:26

you know kind of the virtual and

6:28

one of the very fun things I've seen

6:30

well the first example is obviously the

6:32

Pokemon franchise how it went uh you

6:36

know viral and worked out that way but

6:40

the other interesting example that I've

6:41

seen recently is the stuff that guys at

6:45

vadam are doing so it's like

6:47

v-a-t-o-m and

6:50

they're kind of like this complex

6:52

startup where there are like Virtual

6:54

Worlds and nfts and a lot of other stuff

6:57

but one of the coolest things that

6:59

they're doing is that

7:01

they kinda distribute this stuff the

7:05

nfts and their little objects

7:08

kind of scatter them around geography

7:11

and then you can walk around similar to

7:13

like Pokemon go and try to check them

7:17

out with your app and then you can do

7:20

the same you can take this you know you

7:22

know virtual items or whatever and you

7:24

can scatter them and do like this crazy

7:26

treasure hunt all over

7:29

um how do you feel about the

7:31

possibilities of mixed AR and mixed

7:36

reality in general

7:38

with mobile devices and with

7:41

traditional stuff like The Meta is doing

7:45

in terms of like Hardware do you feel

7:48

like you have to use

7:51

virtual goggles basically to make it

7:54

work 100 or do you think stuff can be

7:56

done with other tools like mobile you

8:00

know mobile devices for example

8:03

that's a good question um

8:05

uh obviously right now using a VR

8:07

headset is the way a lot of things are

8:09

mapped out and that's just because

8:11

that's what's available on the market I

8:13

think that it's only a matter of time

8:15

until we see things like mobile devices

8:17

also having similar capabilities and I

8:20

think that once that's available on both

8:22

your standard Android devices and iOS

8:25

devices that will see a lot of

8:27

developers taking advantage of that and

8:29

like you were giving the Pokemon go

8:30

example I think it'd be very cool to see

8:33

you know like if Pikachu's there on your

8:36

sidewalk that Pikachu is actually on the

8:39

sidewalk and you can move your phone

8:40

around and actually keep Pikachu in that

8:43

you know same location so I think that

8:46

once we see you know those tools being

8:48

offered we'll see developers starting to

8:50

take use of them and I think that you

8:52

know possibilities are endless I don't

8:54

think there's a reason why we have to be

8:56

stuck on VR headsets only

8:59

and um when I saw the trailer for the

9:03

game I was really excited because it's a

9:06

for me because I'm a gamer with uh you

9:10

know coming so I was born in the 80s so

9:12

I've played like a lot of different uh

9:13

generation I saw games evolve

9:17

um it's a very different experience when

9:21

you see this kind of mix between the

9:25

real thing that's around you and also

9:27

the virtual elements there and from your

9:30

perspective as a developer

9:32

why do people like it so much like what

9:34

do you see it does it give more

9:37

engagement to the user does the user

9:40

feel kind of more in the you know in the

9:42

moment when he's having experiences like

9:45

that like what are like the things that

9:46

really trigger with the user

9:57

felt the same thing when they were kids

10:00

um just you know playing with like toy

10:02

guns and weapons like bows and arrows

10:04

just you know make-believe as kids right

10:07

but to sort of have that make-believe

10:09

come to real life right it's kind of

10:12

what this you know mixed reality is

10:13

doing so when you're literally shooting

10:16

zombies in your headset

10:19

um it's kind of like that question of

10:20

how would I survive in a zombie

10:22

apocalypse and with mixed reality it's

10:26

you know very realistic so when you

10:28

physically move around in the game world

10:30

or sorry I should say in real life which

10:33

is the game world now because it's the

10:34

same thing they're mixed you can

10:36

actually Dodge zombies right so if the

10:38

zombie's right next to me in my room I

10:40

can physically move to the back part of

10:42

my room to get away from the zombies I

10:44

don't have to worry about using a

10:45

joystick to move around or change my

10:47

position because it's literally my body

10:50

that affects that and I think that's one

10:52

of the most engaging things about the

10:54

game itself and just mix reality games

10:56

in general is that ability to have like

10:58

this very realistic experience that

11:00

really elevates you know vr's already

11:03

been called realistic but to really

11:05

Elevate that realisticness to another

11:07

level

11:09

Michael so we often hear this when we

11:13

talk about

11:14

mobile

11:16

um

11:17

and VR games especially that this is a

11:20

little bit of a

11:22

isolating experience like the you put on

11:25

your goggles and you kind of hide away

11:28

from the world and even like from the

11:30

outside it does look kind of creepy

11:32

right if you've seen this image where

11:34

where you know Mark Zuckerberg is

11:37

walking around this uh Hall with a bunch

11:40

of people kind of wearing those goggles

11:42

it does look like it's not a very nice

11:44

thing but as people from the industry

11:47

and people who kind of know how it works

11:50

we know that it can be a lot of fun when

11:52

you're playing with friends and it's a

11:54

it's a it's a different level of

11:56

Engagement when you're in this virtual

11:59

world and you're in VR headset in your

12:02

opinion when do we

12:05

start seeing kind of more of those

12:08

multiplayer experiences where it's not

12:11

just me and the headset but maybe it's

12:14

you know my wife was a AR machine like

12:18

it can be maybe just some just glasses

12:20

regular glasses or maybe augmented

12:23

glasses or maybe some mobile phone and

12:25

so on do you feel like there is

12:27

a need for this kind of experience or

12:29

maybe it's just

12:31

that on this platform I want to just be

12:34

alone and kind of work this way

12:36

I think there's always going to be the

12:39

desire for both types of experiences

12:42

um similar to how I believe that will

12:43

always have traditional gaming I don't

12:45

think VR is going to you know deprecate

12:49

you know console gaming slash PC gaming

12:51

I think they'll coincide

12:53

so in the same way I think that you're

12:56

going to have both single player mixed

12:58

reality experiences as well as

12:59

multiplayer mixed reality experiences I

13:02

think with mixed reality being such a I

13:05

guess you'd say it's in its infant

13:06

stages right it's very young it's a new

13:09

technology

13:10

um it's really only a matter of time

13:12

until developers start to try you know

13:14

multiplayer experiences and

13:16

cross-platform experiences like you were

13:18

saying maybe one person has their

13:20

headset on someone else is using their

13:22

phone

13:23

um we've seen some games like that on

13:25

Steam where some players play with a

13:27

game like a traditional game controller

13:28

other players play with the VR headset I

13:32

think developers will have to experiment

13:33

with this in the upcoming years and sort

13:35

of see what works and what doesn't

13:37

um I would love to say I can predict the

13:40

future uh I don't think I can of course

13:42

but I would expect that we're going to

13:44

see a lot of multiplayer mixed reality

13:46

experiences in the future because I

13:48

think that's really going to bring mixed

13:51

reality to another level past where

13:53

we're already at I think that's to me

13:55

one of the most exciting things

13:58

um I have this question and we're kind

14:01

of slowly starting to drift away just

14:03

from mixed reality and more into the

14:05

realm of VR in general and kind of how

14:10

we're going to perceive games because

14:12

sadly we're or not sadly we're kind of

14:15

moving to the end of the year now and

14:17

it's almost September we'll only have a

14:19

couple of and it's time to kind of start

14:22

thinking about what's going to happen in

14:23

the future and you mentioned that you

14:25

can't really predict it but let's try to

14:27

just fantasize a little bit when

14:31

um when you think about this whole

14:32

concept of metaverse and not just in the

14:36

terms of that you know Facebook is doing

14:38

but you know Neil Stevenson kind of uh

14:42

metaverse idea William Gibson kind of uh

14:45

idea of the virtual world

14:47

um from your perspective as a developer

14:49

and the person who's been working a lot

14:51

with kind of this mixed reality Vision

14:54

where do you think we're gonna start

14:57

seeing the first adoption of this High

15:00

concept

15:01

what are the spaces in our world where

15:04

this would benefit they would benefit

15:07

tremendously if they had another layer

15:09

kind of on them

15:12

I think that's like a really open-ended

15:14

question of course and to be frank even

15:16

though I'm developing games for you know

15:18

the quests and the things that are in

15:20

mixed reality I still think the concept

15:21

of metaverse is a little confusing to me

15:24

especially with how the term is thrown

15:25

about around so much

15:27

um and I think uh it's also been

15:30

interpreted in different ways depending

15:31

on who you're talking to

15:34

um so I guess depending on you know what

15:37

sort of question or what sort of answer

15:39

you're looking for there could maybe

15:41

change what I'm going to answer with but

15:43

sort of just start I guess some ideas

15:46

that I would have just for how I would

15:48

like to see this concept of metaverse

15:50

taken in the future is being able to

15:53

really you know again I'm using the word

15:55

Elevate here but to like Elevate things

15:57

that already exist to another level so

16:00

I think in a way in the future if we're

16:03

at the point where people can just wear

16:04

glasses instead of a whole VR headset

16:06

that would be really easy and much more

16:09

accessible for people but I can imagine

16:11

if you're walking around at your local

16:12

shopping mall or maybe you're at like a

16:14

concert venue just having these mixed

16:16

reality elements could probably have

16:19

some sort of interaction between you and

16:21

everyone around you or like at a concert

16:23

you and maybe the person who's running

16:26

the concert

16:28

um just having more ways to interact

16:30

with the real world in this virtual

16:33

world I think that sounds very exciting

16:35

to me personally

16:37

when

16:38

um thank you so much for the answer and

16:40

I know that we can't really predict the

16:42

future here or say what's going to be

16:45

but it's nice to see

16:47

projects like yours where they kind of

16:49

blend this uh border this thin line

16:53

between what's real and not real and

16:55

kind of I think it's a very interesting

16:57

way of looking at it and

16:59

from my perspective a lot of companies

17:01

which are much bigger they're they're

17:04

thinking in the same direction they

17:06

think like okay uh you walk around the

17:09

store and you kind of feed more stuff

17:12

into you I'm not sure if it's a good

17:14

thing or bad or how do you approach this

17:16

but

17:18

um my next question is kind of connected

17:19

with the implementation of this

17:22

technology and the challenges of the

17:24

development so

17:26

when you're thinking about a game like

17:28

yours right when you're thinking about

17:30

this blending of real and virtual what

17:34

are the specific technical skills that

17:38

that you're looking for from Engineers

17:39

or artists is it very different from the

17:43

production of a traditional game or

17:46

maybe it's kind of the same the skills

17:49

are the same and the tools are very

17:51

similar

17:52

I think there's a lot of overlap with

17:54

traditional game development but I think

17:56

it does require a much different mindset

18:00

from the design side so I think that can

18:02

be the most difficult part is when

18:04

you're working in a VR game you need to

18:06

think of it from a design perspective

18:08

differently from a traditional game and

18:10

then with mixed reality you have to

18:12

think of it even more separate from

18:14

traditional games because it's already

18:15

different from your VR games right so

18:18

like I was given the example earlier

18:19

when you literally have to move around

18:21

in your real space you also have to

18:23

account for that and of course on the

18:26

engineering side you do also need to be

18:29

able to understand like you know how are

18:31

we going to implement these mixed

18:32

reality elements and so forth which is a

18:35

little difficult but I don't think it's

18:37

anything uh that you're you know I guess

18:41

mid-level or above software Engineers

18:43

couldn't figure out I think it's pretty

18:45

accessible

18:46

but um artists for example will need to

18:49

optimize there are a little bit better

18:51

just uh because there is that limitation

18:54

of frame rate in the virtual world

18:56

especially when you're mixing in um you

19:00

know pass their elements as well which

19:02

has a little bit of frame rate overhead

19:04

so just think about like you take your

19:07

traditional stuff and you have to make

19:09

it a little bit better essentially to

19:12

handle this mixed reality space but I

19:14

will say I think it's still accessible

19:16

enough for the average developer I

19:18

wouldn't say it's too different

19:21

do you feel like we're gonna still

19:23

because a lot of this optimization stuff

19:25

is still kind of connected with the

19:27

hardware requirements that we have in

19:29

the both and like headsets and the

19:32

mobile devices although they are made

19:34

becoming much more powerful they're

19:37

still kind of lagging

19:38

behind you know powerful stations that

19:42

you could have at home and my question

19:44

is do you think there is another way of

19:47

just kind of rendering the whole thing

19:49

do you do you feel like we'll see more

19:52

streaming Technologies taking place or

19:55

do you think maybe some

19:59

older Technologies might pop up so let's

20:01

say you know webgl or something like

20:03

that do you feel like like it's a viable

20:06

option when you think about the delivery

20:08

of those experiences or do you think

20:10

it's still just going to be client

20:11

server and when you need to build a game

20:13

and it has to run on some Hardware that

20:16

you have on a platform

20:18

uh I'm not sure if I fully understood

20:20

the whole question there but uh I I'm

20:24

sorry could you actually repeat that one

20:25

more time I just want to make sure I'm

20:26

answering it yeah like when you're

20:28

building this VR thing right

20:30

um

20:31

do you feel like you could deliver it in

20:33

a different manner like for example you

20:36

can stream the game and then it will be

20:38

available on a larger number of devices

20:41

or maybe you can do a webgl client and

20:45

you know like a webgl page I know it

20:47

still requires some Hardware but

20:50

do you feel like it's a viable option or

20:52

do you still think you know just build a

20:54

game let the player download it optimize

20:56

it to a certain

20:58

preferably the same kind of platform so

21:01

it's not really a lot of variety what is

21:03

like the the better option from the

21:05

developers then so

21:13

download capability to be an option

21:16

there are a couple other sorry sorry

21:18

when I say download I was actually

21:20

referring to streaming I don't see

21:22

streaming as being viable for this

21:24

um and there's a couple reasons for that

21:26

I think the biggest one is the input

21:28

latency

21:30

um that would be really big in a VR game

21:32

and that's going to take a long time I

21:34

think until we see very very efficient

21:36

streaming options for games especially

21:39

on VR headsets

21:41

um and as you may know like there's

21:42

definitely issues with nausea for VR

21:46

games I believe that would also probably

21:48

correlate with Mr

21:49

but um another one being that things

21:53

like the pass-through which currently

21:57

is all handled on your own device until

22:00

the streaming options have like a back

22:02

and forth setting for game streaming

22:05

um I don't know if that would be

22:06

functional so so just give you a little

22:08

bit of example here

22:10

if we're running all of this passive

22:12

data on our own device that would then

22:16

have to be passed up to wherever the

22:19

game is running from on the server and

22:21

then the game day would have to be

22:22

passed all the way back to the client

22:25

um and then we have to sort of render in

22:27

a specific order which just might not be

22:29

feasible for game streaming at least at

22:32

this point in time it also sounds like a

22:34

little bit of a headache from a

22:35

developer standpoint I do think the

22:37

simply just download the game onto your

22:40

device is probably our best option for

22:42

now at least for VR and Mr games but I

22:45

think you know

22:46

like we said you know we can't predict

22:48

the future so we might have some good

22:49

stuff coming out in the future but we

22:52

won't know until it happens

22:53

all right so it's time for our uh flash

22:56

pull so

22:58

um I'm just gonna start and we'll see

23:00

how it goes

23:03

all right so

23:05

uh what's your favorite game of all time

23:09

uh Super Smash Brothers

23:12

so what are like the most underrated

23:15

games that you can name

23:16

yeah

23:18

I know it's a flashable I should think

23:20

of something faster I can't think of

23:21

anything off the top of my head

23:23

um

23:25

Android is a strong word here

23:30

I really enjoyed the uh game Fantasy

23:33

Star Universe it wasn't amazing but uh I

23:36

enjoyed it a lot that was MMORPG from

23:38

mid-2000s

23:41

what in our uh video game industry uh

23:45

fascinates you what are the things that

23:47

Fascinate You I think just seeing the

23:49

creativity from all the different

23:51

developers out there and constantly

23:53

seeing new I guess new experiences is

23:56

what I'm looking for

23:58

what's the coolest or funniest fact

24:01

about you as a gamer

24:03

uh I barely play games anymore because I

24:06

spent so much time making them are you

24:08

like uh are you like a sore loser when

24:10

you're playing a Counter-Strike or

24:12

something like that no no I'm not

24:15

I used to be a little bit worse

24:19

what did your parents think about video

24:21

games and your career choice uh they've

24:25

always been supportive of me actually

24:26

which I have to be very thankful for

24:29

I I think it's also

24:31

um kind of a mark of time because I

24:34

guess when my mom thought me playing

24:36

video games just thought I was just

24:38

wasting time

24:39

um

24:40

uh so they're kind of supportive and

24:42

they support you now as well

24:44

um

24:45

so if you could do one job

24:48

to the end of your lifetime what would

24:51

it be

24:54

game development honestly just making

24:56

games that's my favorite job I've ever

24:58

had and it still is all right well thank

25:02

you so much for joining us Michael it

25:03

was very fun talking to you

25:05

um I will leave the links in the

25:08

description to this video and you can

25:10

check the website for more links to your

25:13

game and if somebody wants to join your

25:15

team maybe they can send you an email or

25:18

contact you on social media or through

25:19

your website so you can maybe find some

25:21

more teammates alright thank you so much

25:24

thank you girl it's been great thanks

25:27

for enjoying another episode of the 80

25:29

level Roundtable podcast

25:31

upcoming episodes on the 80 level

25:33

website at 80. LV join our career site

25:37

at 80.lv RFP

25:41

and share our podcast with friends and

25:43

on your social networks

25:47

[Music]