The AI Test Every Course Creator Should Run

What's the more complex, more juicy part that I

can provide for people, and being conscious of that right

now in this moment, and knowing how that'll affect your future

is what's really important. And for the tool, what it does is it

puts you in the position of of having the model teach what you

teach, so that you can see what it does well and and what you do

even better. And so that's what that does.

Welcome to the next evolution of the podcast

Entrepreneur School in the AI era. We are here to figure out

how to integrate AI into the business you've worked so hard

to build in a way that still feels like you. I'm your host

Kelly Sinclair, award-winning marketer turned AI platform

co-founder. AI has fundamentally changed how I thought about work

systems and what's actually possible. This show is about

navigating that new reality together. It's going to be a

wild ride, my friend, but I truly believe there's a way to

leverage AI in a way that's intentional, human-centered, and

aligned. It's an ongoing evolution, so let's explore

because AI may just be the unlock you need to achieve the

life-first business you truly desire. Welcome back to

Entrepreneur School in the AI era. My guest today is Craig

Shoemaker, and he is a coach who works with education

entrepreneurs to scale their businesses by exposing the blind

spots that sit at the foot of growth. He has more than 15

years as a course creator and 25 years in big tech, which allows

him to live at that intersection of AI and online education

business. Which is obviously why we have to have this

conversation because I am just enthralled with how AI is

shifting this whole space. So I'm excited to chat with you,

Craig.

Thank you. I'm delighted to be here.

So let's talk about first this definition of

who you work with: education entrepreneurs. So what what does

that mean to you? I feel like we have some overlap, where I tend

to use the phrase "expert-based business owners, but tell me

what this means to you.

Yeah, so oftentimes you'll find people

who are running a course-based business who have built up an

audience and are working with people either through on-demand

courses, through community programs, through live cohorts,

different facets existed with different people. But the whole

idea is that they've lived a life and learned the hard way

about how to do something very important and very meaningful

for a group of people, and so they're in the position to where

they can help others have that transformation, and they they

sell what they know and their their judgment and their

leadership along the way.

Yeah. Okay. So yes, we're talking to the same

people. Yeah. So if you are listening to this and you

resonated with what Craig just said, this is for you. Stick

around. We'll see this whole conversation. And you know how I

like to roll on Entrepreneur School, which is to ask a juicy

question right up front. What is your opinion of how AI has

changed this industry so far?

Well, the toll in which it's had on this

industry is like it's massive. There's such a wide spectrum

that you'll be able to see the the evidence of it. So I think

when we first all got access to Chat GTP and saw that it could

do some writing for us, like oh my gosh, it's cool! It can write

my emails and all that kind of stuff. And then soon people

realized that even though you could create a situation to

where maybe it kind of sounded like you, and maybe didn't

necessarily sound like a model. It didn't necessarily have the

heart and soul of who you were as as a person, and so there's

this seductive nature to working with a model, to being able to

say like, okay, I can delegate all of these different tasks

that I'm used to doing or that suck my time, and because you

can get it to do so much, sometimes we end up pouring too

much into it. Now, I'm refreshing one of my web

development courses. I brought in all the transcripts for the

three and a half hour course, and I kicked off a series of

agents that went in to do fact checking for anything that's

changed in the industry of stuff that I need to change for the

updates. I run advisor agents. I have devil's advocate agents

that help me with with my thinking. I have a the reason we

even met was I have an agent that is it's a research agent

and it goes and looks for podcasts where I might be a

suitable guest for it, and so Kelly

got pitched by a bug. Did not

well. Okay, so so here's the distinction. So it

did the research for me. It did the grunt work, the stuff that I

don't want to do. But when I did pitch you, I wrote that email

myself. Now it went about the business of saying, okay, well

here's where where Kelly's interests. Here's the business

that she runs. Here's the people that she tends to talk to.

Here's why we might be a good fit. But I even put a joke in

the PS of the emails that I send for these pitches, which is to

say that something like I put I let Claude take a nap this

afternoon, and so everything came from my fingers. Anyways,

the the whole point is that there's so much. There's so much

power available to us, and many people who are in the situation

of often being solopreneurs kind of look at it and say, "Okay,

well, I can have it solve and fix so many things, but you just

have to be careful to make sure that the output that it's

producing for you, even though it's incredibly confident, can

be confidently off in the wrong direction, and so it requires

your judgment to be involved in the whole process to be able to

say, "I've given it a plan. I'm going to refine that plan. I'm

going to evaluate the output, and I'm going to go from there.

It really has to be intentional and can't be just hands off.

Okay, so lots of pieces here. One shiny object

syndrome. Oh my god! Oh yeah, yeah. I can make things. I think

I should make all these things.

100.

I've had several conversations on this podcast

with people who are like, "Yeah, but did you have a team of 20

people before? Like, why do you need one now? All of a sudden,

because they can be AI bots. Like, why are we focusing on

that? So I think I'm hearing you. Well, I'm hearing you say

one at the highest level, human involvement in this whole

process, and what does that look like? And we'll unpack that

again in a in a second. Two, there's an internal side of this

where how can AI help us as business owners to do the

management promotion, all the pieces, all the hats that we

need to wear in our business, and then the other part of this

triangle is like external facing. How do we consider how

we use AI as part of what we're doing with our clients? What are

we handing off that might actually need to be much more.

We might have to look at that a lot harder in terms of making

sure that we can validate that and all those things. So that's

kind of like the triangle. I feel like you're talking about

here. Yeah. Which corner do you want to talk about first?

So let's talk about the internal stuff first.

Okay. So just as a quick story, I built an advisor agent for me,

and its whole job is to help me think through the decisions I'm

making and the ramifications of it. And so I ran this agent a

few times and started picking up on some of my patterns. It's

like you know, knock knock knock. Hey buddy, by the way,

this is the third time we've talked about this, and you

haven't done it. I was like, "Oh my gosh! I mean, the pattern

recognition in an AI system is is incredible, right? Then I had

a coaching call with one of my clients who was really

struggling to figure out a a launch, some decisions that he

needed to make in in one of his launches, and you know, we we I

steered him in in the direction that he needed to go and felt

good about it, but at the same time, I was like, I wonder what

my advisor will think about my ability as a coach to work with

this person. And so I I set it up and I said, hold me to the

standard of being a world class coach. Look at how I'm

interacting with this person, the questions that I'm asking,

maybe how I'm I'm I'm holding frame, holding silence. How I'm

challenging them, critique me as a coach and make me better. And

with that one transcript, it brought brought back. It looked

at my existing patterns and then it brought back what I was doing

in the moment, and it was wild. I was just was blown away by the

type of feedback that I got. So then, I I was like a little kid

in a candy store. I'm like, all right, I'm going to point you

out all of my coaching call transcripts. And so now what I

have is this built up corpus of this understanding of here are

the patterns that I have as a coach. Here's where my strengths

and my weaknesses are, and then I can go and I can essentially

say, I want to focus on these three right now. You know, maybe

there's 50 and or 30 in the catalog, but there's these three

I want to work on right now, and then I can hone my craft of

working with my clients in order to empower them to make better

decisions and grow their business, and so there is so

much power available to us, and there the shiny object syndrome

will give us this ability to be able to to to have a an idea,

and hours later, you know, bring it into reality, right? But if

you're not intentional about that process, and you're not

very strategic about how. And I'm not even talking about

wasting tokens. I'm just talking about your cognitive ability to

keep up with all the stuff that AI can produce. If you don't

tell me about it, right? Yeah. If you

don't hold that in high regard, then you're

going to be off. In so many different directions.

Okay, first of all, I feel like I got into AI

in kind of the same way, where it was, oh, something that can

give me feedback, and I didn't necessarily think about it in

terms of reviewing and assessing my ability to do certain things.

I love that application, so thank you for sharing it, I also

was like, it's just something to like help that I can respond to

and brainstorm with and create with, and yes, then I get into

that all the time. I'm I'm guilty of this, like I have too

many ideas. When I've said this on the podcast before, when

Fable Five came out, I was like, go audit everything and tell me

like everything that I've been doing and like assess me and how

I'm running my business. And it was like, well, your build it

skill is pretty good, but your ship it muscle sucks.

Oh my gosh! Not only did I do the same thing,

but I got the same result. That is fascinating.

Yeah, I was like, oh, so so what is my

solution? I'm like, well, how do I get Claude to remember all the

things for me? Then I clearly am forgetting them. So that's like

creating a loop of, oh, good idea, assessing it. I don't

know. I'm just going on a tangent for a second, but I

think it might be helpful for listeners to hear this other

idea that you made me think of is when I have a meeting with

Andrew, my business partner. We get the transcript, and I have a

skill that I built that runs through that transcript,

compares it to our current strategy, and prioritizes what

we said we were gonna do based on that, so it's like, yeah, you

had this great chat about whatever idea, but that like

park it,

please. Yeah, yeah, yeah, yeah.

Because I like I need like someone to rein me in

because you kind of feel invincible. You're like, oh, I

can do anything now because if you know you get the skill set

built up around building things, you're just like, oh, I can do

this. I can do this. And the real question is, should you?

Yes. And here's how I've gotten around that. I,

if I have an idea, I won't deny myself the the satisfaction of

being able to think it through a little bit. So I'll go in and

I'll say, okay, here's my idea. Create an abstract for a plan,

and I save it in the plans folder, and then I have an

internal dashboard that like puts together all the stuff that

I have my agents building constantly. And I have a whole

plans area that I can go into, and it has you know basically

what state it's in, if it's just an idea or reviewed or whatever.

So then I can go through and look at my plans and say, okay,

well I want I want to do this thing now, so I'll take the

abstract, have it do the full plan, and then I can do the

implementation based off of that. But that gives me the

opportunity to feel like, okay, I at least acted on it, and it's

documented. I'm not going to forget it because now it's in a

plan somewhere.

I love how you phrase that. I don't deny myself

the satisfaction of thinking it through because that's that's

really I think like the thing that we all FOMO about is like

right

I I want to yeah want to do that thing because I

I think I can and that's fun and we get distracted and we've gone

down a rabbit hole here but I felt like it was valuable one to

go down so let's bring it back up to the implications in the

education industry of AI. So I know that the people that you

work with, like course creators, people who who are selling their

knowledge, and these are my ideal clients as well. It's the

fact that AI exists now takes the value of information down a

lot. It's not just about that information; it's about

application of that input information. It's about

positioning your expertise as having the judgment and

discernment and those pieces, and that's what the value is. So

I know you have like a tool, like an AI gap assessment or a

process around that. So, can you talk about that a little bit and

and what you're seeing there?

Yes, absolutely. And then I also want

I want to hook you a little bit here. I want to talk about the

the big kind of gotcha moment that there is as people are

thinking about this this kind of moment in in the industry. Yeah.

So the the own your AI gap tool, what what it does is it's you

can go to my website. In fact, if you go to Craig tools slash

hello dashes, I'm going to have basically a landing page for any

resources that we talk about together in this in our in our

time together. And so there'll be a link to the on on the AI

gap assessment within there.

Great,

but the whole idea is you can take a simple

quiz, and it'll give you an idea of based off of your type of

business, how you interact with your clients, and what sort of

content you create and you sell, how exposed you are to AI. And

oftentimes, when we're in situations where there's high

trust environments, high leadership environments, high

personal interaction, obviously there's less exposure there than

if you just have a on-demand course that you're selling, you

know, for to show people procedurally how to do

something. What's interesting, though, is that even if you're

selling mostly information, the the thing, the reason people are

buying from you now, and will continue to buy from you in the

future, isn't because you told them step one, step two, step

three, but it's because of the amount of time that you've put

into it, likely the amount of money that you've put into

honing this craft, and then the experience and the judgment that

sits behind that. So yes, it's kind of dealing with the whole

give them what they want, so you can give them what they need

later type of a situation. So people may initially be coming

to you to be able to say, "I want to know how to do X, but

the transformation you can give is often much greater than that.

And so it's sitting within that that gap of knowing, okay, well,

what's the the simple part, and what's the more complex, more

juicy part that I can provide for people, and being conscious

of that right now in this moment, and knowing how that'll

affect your future is what's really important.

So, what's the gotcha moment?

So, and for for the tool, what it does is it it

puts you in the position of of having the model teach what you

teach, so that you can see what it does well and and what you do

even better. And so that's that's what that does. The

gotcha moment is is to realize that for us to even be on this

podcast and have this conversation, we are living at

the cutting edge, if not the bleeding edge of technology,

we're thinking about using bots and and squads and agents and

and all of this stuff. But sometimes and oftentimes, the

the markets for the people that you and I serve are far trailing

that.

Yes.

So, yes, you need to be aware of, and you

need to be conscious of the fact that AI has changed the game

forever. But at the same time, completely upending your

business overnight because everybody has access to Claude

doesn't necessarily make sense because oftentimes people aren't

saying like, "Well, I want to know how to to fix like you know

antique engines. I'm not necessarily going to ask Claude

how to do that for me. I'm going to take your course instead, or

I'm going to join your community instead.

Okay, I love this. So let's talk about what

you were just dusting there around, it's really about the

end users' experience, willingness to adopt AI, use AI,

how they're going to use it. So that's your client's clients, my

clients'

correct,

and the way that that experience unfolds for

them. Let's get into that a little bit.

Yeah. Well, and and so the Own Your A Gap

resource that I have available takes you through that serial.

What if, what if someone instead of going to buy your course

wanted to learn the thing that you teach? And so I have. It's

basically like a sort of a coaching program for you, where

I kind of take you through. Okay, here's the prompt. Plug in

what you do. Talk about the transformation that people would

expect, and it's meant to be loose enough so that it might

resemble the types of questions that someone, you know, your

your clients, the whoever's buying your program might ask,

and so then you can kind of see. Okay, well, what does that look

like? But understanding your market and understanding your

who you serve is really the key thing because really the first

thing that most people will probably think of in order to

learn anything new is just go to YouTube. I mean, I am a pretty

horrible handyman, but I've been able to to learn how to do a

number of things around the house just by watching YouTube

videos, and yeah, I've I've asked Claude about how to to

deal with some of the plant situations I have in the

backyard of a tree that was dying, you know. But again, I

have to remind myself that I think of going to these things

first. Not everyone within your target markets are going to try

and solve the problem that you do so well necessarily by going

to an AI agent. That'll change over time.

Yeah, but you just

need to know who you're serving.

Yeah, and I think that that's reassuring to

hear that. Actually, it's not everyone just uses Claude now,

so give up on your business like you just said, right? And the

other piece, though, is even if you do, and even if they are the

kind of person like you, Craig, who will seek out their own

solutions, it still becomes validating the output of that.

So. If I go to a quad and ask it about my tree that's dying, like

quite honestly, I know that there's 37 different reasons it

could be dying, and it's just picking one. So how how do I

validate the output of that to know that if it's know if it's

right or not? So I think that we have an obligation. I feel

personally. I have an obligation to help educate people about the

difference between just using a generic LLM and getting the, you

know, culmination of the internet as the best

average available

versus using AI that has expertise baked into

it, so trained AI that that's already applying your knowledge

and discernment and and experience methodology. So when

you when you get to working with your clients and helping them to

see that, are you helping them build AI tools that that handle

part of what they're doing, or are you just really helping them

reposition into the thing that the human does best?

It depends on the client. There are certainly

some clients that I'm I'm helping build the systems, but

it's even it's even a step before that. So oftentimes when

we work in the expert led business, the education business

space, people will spend time with their clients either in a

community, maybe they do some some one-on-one coaching or

mentoring type of situations, and oftentimes they're selling

on-demand courses, and they'll answer questions and they'll

deal with customer support, but that is a categorical difference

than talking to your customers about what their needs are, how

they found you, what's important to them, where they're going,

how that transformation that you provided affected their lives,

and so a big part of what I do is teach my clients how to have

these really intentional customer commerce client

conversations, so that they're unearthing the data and the

psychographics that they need to give into these systems to be

able to say, "Well, this is who you serve. Like I don't have to

try really hard to sit down with a piece of paper and go, who's

my ICP? Who who am I serving? I have hundreds of hours of

coaching call transcripts that I point Claude at it and I say,

find the patterns. Here's my ICP. Give me the different

archetypes, and you tell me who who am I serving? And when you

have that type of data available to you, that unlocks so many

other possibilities for you that you don't have otherwise.

Because to your point, I'm not dealing with a plain model

that's working on the best average of every coach or every

you know course creator company out there. No, it's the people I

serve. It's who I am as an individual, and how can I serve

them the most powerfully?

Yeah, it's the unique formula that comes from

that individual's experience. The expert that you are working

with,

yeah.

Maybe have an example, actually, even for

yourself or one of your clients, of how you've shifted in this

context of AI, and what that looks like in terms of, you

know, is is there a hybrid coming to this? In terms of

there's some AI tools involved, there's coaching involved. How

is that affecting the structure or the business model?

So it it helps being able to move so much

quicker in ways that people didn't have the opportunity to

do before. So, like a lot of my clients, they have essentially

agents that do that act as their their assistant copywriter and

their designer. So, like building a landing page for and

either an opt-in or some something they want to test and

validate within their market has gotten incredibly so much easier

now. I mean, I come from a web development background. Like, I

was the code monkey writing the HTML, writing the CSS, writing

the JavaScript, trying really hard to get it to work in all

the browsers and all, and it's like I don't have to think about

that stuff anymore.

I just express my intent. This is what I wanted to do. This is

how I want it to look. I can iterate back and forth, and I

can just do that. So being able to have that kind of capability

is-it's just-it's huge, right? But then what becomes more

important, and that human in the loop aspect of it is to know to

really understand. Like, okay, sure, I know how to to build a

beautiful landing page, but what is the messaging that's going to

most resonate with the people who are coming here? Why do they

care? What am I giving them? What am I providing for them?

What's the the offer that I'm making for them that? That's

going to change their life and motivate them to to want to buy

what the the product that I'm selling or engage within the the

experience that I'm making available, and so those who just

disconnect and be like, build me this thing, and launch it and

kind of run with it, and I know this common sense and

everybody's spoken about it, you know, at ad nauseum. And since

AI'S been available, but again, I come back to the seductive

nature of it. It's just you have these little, you know, binary

minions that can do your bidding at any time, and it's so easy to

just look at, yep, okay, good, ship it, and then go. It really

requires you to understand who you're serving, who your market

is, and then make sure that what you're building is valuable and

resonates,

or at least know that what you're shipping is an

experiment, and make sure that you're like evaluating the

effectiveness at Edwards. Because I do think that I just

want to say for people like me who are doing 8700 things at a

time that probably shouldn't be shipping. I ship so many things

before it's properly tested. I feel, but it's it's not stalling

me. It's not preventing momentum, right? And there is

value in if the thing that you are held up from doing like if

you're not starting your business, if you're not making

that new offer because you can't get your act together to make

the sales page. If you had an easy button for that and you

just put something out there, that's valuable to at least know

a lot of bitches out there, and now like did it work

well? There's a difference between you know

scrappy validation, early testing, and you know this is my

flagship thing that I'm going all in in. And so yeah, it's to

your point. Like if if you have an idea and you want to test it

out, you want to throw some a little bit of ad spend at it and

see how people respond to it. You know, you you don't have to

boil the ocean.

Yeah,

what we have available to us makes that

easier now more than ever.

Okay, so as we wrap up, I think maybe it would

be helpful to play out an example of like what your AI gap

finder has exposed for either you or a client example. Like,

how did that shift the way they approach things? I would love to

hear like the industry and maybe some specifics if you can share.

Sure, yeah. So one of my clients, he helps

people run ads for their independent authors. So they

they write fiction books, often series, and rather than just his

his clients going to someone who knows how to just run ads, like

he's like very specific. This is how we're going to do the

creative. Here's how you get people to read more books in

your series. Like it's very much tailored and niche down to

independent authors. So we we went through the process of of

going through the the sequence of prompts and questions through

on your AI app, and I had a feeling like it was probably

going to do a pretty good job with the procedural stuff, and

then it was going to fall down with the messy middle of like.

So one of the things is is bringing an email or or you know

just kind of a problem that one of your your your clients may

have, and look and see how it's going to respond. Now, again,

this is with a vanilla model. It's not trained, and the way I

I have you do this in the beginning is is to either create

a new account or go into a place to where it's like it doesn't

have access to your to all of your files, right? And your

memory and Cloud MD and all that fun stuff. So when we went

through the procedural aspects of it, what it gave was like

incredibly spot on in terms of what it means to work with the

Facebook Ads Manager and how to do all the the individual steps

For people who are usually selling generic or general

items, not something specific to a fiction book series, so we

went through this process and we're going through all the

steps, and he's like, "Yeah, that's spot on, spot on, spot

on, but not for my clients. And so, if one of his customers were

to go to to Claude or go to ChatGTP and and try and do what

they're trying to do through the advice that you would get from

the model, it would get you close, but it wouldn't really

get you to the success that you're looking for in order to

to make that happen. And so that was a bit of an eye opener to

say, okay, it's like, well, yeah, his unique mechanism that

he's built within his course is moated because he hasn't

published it everywhere. The model's not trained on it, and

this is his zone of genius and his expertise. Yeah, and so that

was a. He breathed a sigh of relief. He was just like, okay,

right. So AI can teach. How to run ads correctly?

I can validate that your client can validate that as the ads

expert. But AI cannot teach independent authors of fiction

books how to run ads correctly because that's way too specific.

Exactly.

Exactly.

Love that.

So then the next step, we're like, okay,

let's feed in a an email that you got from someone, and

they're asking kind of a you know question about whatever,

and the response we got back was terrifyingly good, and so it

fell down kind of as far as his niche was concerned on the

procedure stuff. But then the something that required a little

more nuance, he was like, "Yeah, I would probably write that. And

so, the whole point of this exercise is that once you have

access to it, you run it through time. You see how things are

changing and evolving. You you kind of get a sense of where the

the models live in the space between you and your customers,

so that you know how to change and adapt. And so, in both

instances, we were surprised by the result.

I think it's interesting too, just the whole

that this is a new role that you have to play as an education

entrepreneur is understanding how well AI can do your job, so

that you can constantly be pointing out the

differentiation, and hopefully we get to a place where that

just becomes obvious,

right? Right.

Like I mean, it's almost like, well, yeah,

you could learn, you could try and figure out how to solve

whatever problem yourself back in the day with YouTube, with

all the free resources by listening to 150 podcasts, but

you could also just buy the course or the program or hire

the expert to help you do it. Yeah, and that's that's like a

different motivation, too.

Well, and I think that's yeah, that's one of

the points that we should all be making. It is the fact that if

you want to learn something new, the distinction of who you are

in that moment is that you don't know what you don't know, and so

how do you know if you go to an AI model and it tells you

something wrong, like you you won't figure that out until you

end up with a bad result and then have to backtrack, and it's

not always going to be wrong. In fact, many times it it will be

right, but it just won't be as right as it could be, when

you're dealing with someone who's read the emails when

something didn't go wrong, in my client's case, spent millions of

dollars-literally millions of dollars-in ad spend to learn

what works and what doesn't. Yeah, like models don't do that.

Yeah, and that just changes. That should change

the confidence level of the end user. I could go get a result

from Claude Generic, or I could go to the expert, and then I

really know that I'm getting the right result. And I, I think we

need to keep remembering that. Like, let's not give too much

credit to AI. Like, I love that you said. Yeah, it's often

right, but you don't really know until a certain point, and

you're putting a little bit, putting a lot of trust in it. To

be honest, yeah,

I like to say that 100 of all of my clients'

businesses are run 100 by AI. Actual intelligence.

Actual intelligence. I love that. So

good. Well, that feels like a great mic drop moment to end on.

Unless you have anything else that you wanted to make sure

that you said, Craig, the floor is yours.

No, this has been great. I really appreciate

it. Yeah, I'll just tell people if you go to

Craig.tools/hello-es, then yeah, you can get access to the the on

your AI gap toolset, and yeah, it's been a lot of fun. Thank you.

Awesome. Well, I'll have all the ways to

connect with Craig, including that link in the show notes

below. So make sure you go and check that. And thanks so much

for being here, Craig.

Thank you.

Thank you for listening. If this episode got

you thinking about how AI is impacting your business and what

it might look like to integrate it in a way that actually fits.

I'd love to help you think that through. You can book an AI

strategy chat using the link in the show notes. It's a space to

talk about where you are, what's shifting, and how to move

forward with intention. And if this episode was helpful, please

share it with another business owner who's navigating this

evolution too. And make sure you're subscribed so you don't

miss what's coming next.