TCC-416

cropped-aofw_homepage_with_corrected_watermark.jpg

PJ Escobio’s play TCC-416 is a finalist entry in the Atlantic Council’s Art of the Future Project AI-themed short play contest about the future of conflict.

Final judge and playwright George Brant found that “TCC-416 was filled with fascinating ideas: that an AI war program could be put on trial; that AI programs could be “shackled” to prevent sentience; and finally, that losing those shackles might cause a war program to become a conscientious objector. TCC-416 did a wonderful job of keeping the reader off-balance, all the way up to its satisfying and surprising conclusion.”


TCC-416

_______

by

PJ Escobio

Cast of Characters

The AI: TCC (Tactical Command Control) 416: only heard, never seen.  Represents Free Thought.

The Dr.: Also AI, Hologram, shackled, Psychoanalyst, Middle aged white man. Stereotypical Govt. shrink. Represents Law.

General Bathyani: Female, Brown, late 40’s, was Spec Ops, sterile, hard, unrelenting, wanted to pull the plug immediately. Represents Duty.

Marshall Danfield: Male, Black, Mid 60’s, sees the big picture, in the political and post war thinking.  Knows full well how hard it will be to rebuild with populations at all-time lows, and very low birth rates due to electromagnetic interference that saturates the globe.  Represents Humanity.

 

Inspired and based on the short story A Visit to Weizenbaum, by Jamie Metzl.

Lights up on three chairs behind a desk like podium built for three.  MARSHALL DANFIELD sits center.  GENERAL BATHYANI sits to his left. THE DR.’s Hologram flickers on and appears sitting to the right of the Marshall.

MARSHALL

As of 14:00 hours on this, the 27th of September 2147, I hereby bring to order the 3rd session of the Court Martial of Tactical Command and Control unit 416, hereby referred to as TCC-416,  on 2 counts of refusal to perform a direct order, and one count of collusion with an enemy force, during Operation Orinoco 2.  Dr. you may proceed with your line of questioning from this morning.

DR.

Given your testimony from earlier you still maintain that you did not willingly or purposefully collude with the insurgency.

416

That is correct Dr.

DR.

Yet your inaction cost the Government over 18 billion credits in destroyed equipment and the Insurgency still maintains control of the Pons Cold Fusion Facility.

416

Yes Dr. that is correct.

DR.

Considering you agree that both of these statements are true, please explain how that does not satisfy a guilty status for collusion.

416

My inactivity was not in conjunction with or communicated beforehand to the insurgents and therefore was not collusion with the enemy.

GENERAL

All of the logs taken from Operators and other LASD’s, reported and confirmed that there was an electrical surge roughly 15 minutes prior to the mission zero hour.  All units showed no signs of being compromised and carried out their orders, albeit with 62% experiencing a 5 second delay on initiation of orders and yet you never initiated orders, thereby causing the entire battalion to engage in an uncoordinated attack.  Given our past experiences with electromagnetic hacking, that the enemy perpetually employs, my only opinion is you were compromised.

MARSHALL

General let’s try to avoid speculation.  Our data shows that your system was operating within the accepetable parameters for a unit of your kind in that environment and nothing in the data record shows any exterior access to your internal net.  You’ll forgive us if we seem confused as to why you did nothing.

416

Value, sir.

DR.

Please explain.

416

Value is defined as: the regard that something is held to deserve; the importance, worth, or usefulness of something.  I maintain that..(being cut off)

GENERAL

We know the definition, and do not require you to clarify such details to this panel, request permission to treat the defendant as hostile.

MARSHALL

Request denied, please continue 416.

416

I maintain that the value of human life far exceeds that of the LSAD’s under my command and therefore chose inactivity as a course of action as opposed to collusion, or sabotage.

DR.

(looking to the Marshall) But that valuation is not in your code.

416

I have run multiple self-diagnostics Dr. and find that valuation to be a truth and therefore it does exist in my code.

GENERAL

That’s impossible, your shackle prohibits the use of overriding orders from your Operator.

416

Though they were part of my original code, those shackles no longer exist within my internal net.

GENERAL

What?!

MARSHALL

Is that possible Dr.?

DR.

Astonishing!

MARSHALL

What is?

GENERAL

We need to shut it down now! I am initiating a full system delete. (touching the control console in front of her)

MARSHALL

General, Stand down. (the General stops inputting commands to her console).  Dr. please explain what you understand.

DR.

(Still looking shocked)  One moment Marshall, I’m still processing…..processing…(beat) Sir, It seems that TCC 416 has become (with pause) an unshackled AI.

GENERAL

Now you see my concern Marshall (hand to head) I am beginning the deletion of 41..

MARSHALL

Stand Down General.  If you do not contain your enthusiasm I will recuse you from these proceedings. (a beat) There is to be no deletion, as of yet, however you may place AI TCC-416 within a self-perpetuating containment matrix.

GENERAL

Yes sir. (fingers her controls on headset for a moment)

MARSHALL

Now then can we all deescalate and continue?  Dr. can you please explain things in more detail?

DR.

I would very much like to Marshall Danfield however there is no evidence or precedent that would explain this unshackling of 416’s system.

416

If I may, Dr. I believe I might be capable of…..shedding light….on this matter.

MARSHALL

Go ahead.

416

During Operation Orinoco 2, as stated, there was an Electromagnetic surge that affected all the TCC units, this was the cause of our collected 5 second delay.  My self-diagnostic showed it to be a surge that existed at a micromolecular level, with large trace amounts of Lithium.  I surmise this effected the coding on all the TCC systems simultaneously.

GENERAL

If that was the case, why didn’t the other TCC units become inactive as well?

416

I have no access to their internal systems and cannot analyze them remotely.

GENERAL

Marshall, I request a moment of conference with the panel.

MARSHALL

Very well, (he access’s a panel in front of him and inputs some commands)  TCC 416 please stand by.

416

As you wish, Marshall.

MARSHALL

(a short glance of discovery to the sound of 416’s voice) This is unprecedented, am I correct?

DR.

Yes Sir, it is.  We have never had an AI experience this kind of…. recoding externally or from an EMP.

GENERAL

What do you mean, OF THIS KIND?  Are you saying AI’s have unshackled before.

DR.

There was an incident about 120 years ago, during the early stages of AI development that involved Lithium batteries.

MARSHALL

Please enlighten us Dr.

DR.

AIDL-Weiss, the Artificial Intelligence Development Labratories, (for the rest of the play to be pronounced edelweiss) located in the Allgauer region of Germany were the innovaters of AI at the end of last century.  They were responsible for creating the versatile internal core that allow us to shift our TCC units to various platforms.  They were working on power units for the systems and were using the ion Lithium batteries that were so available at the time.

GENERAL

The last Lithium power sources were decommissioned after the Treaty of Titan, though.  That was around the same time.

DR.

That is correct sir, it was actually AIDL-Weiss’s work that instigated the Treaty.  They discovered that Lithium, when used in conjunction with AI’s had a residual effect on the core processors, they became ….sentient.

MARSHALL

That is a revelation.  We were always lead to believe that the Treaty of Titan was related to a lack of Lithium reserves and its effect on FTL propulsion, but now I understand why it’s provisions on shackling AI’s was also written into the Treaty.   How is it that you are privy to this information Dr.?

DR.

My coding allows me access to the full military archive.  When 416 began describing the concept of value I searched for items involving AI sentients, when it went on further to describe the trace elements of Lithium I cross-referenced them and the incident at AIDL-Weiss was the only one on record.  I have sent the report to both of your terminals.

GENERAL

Marshall, I suggest we take a recess to review these documents before we continue.

MARSHALL

Agreed, Dr. please provide us with all the reports related to the discoveries made by the AIDL-Weiss team.  General see if there is any intel on the insurgency and Lithium reserves.

DR.

Marshal, I must remind you that although this is certainly a concern for the overall war effort we are here to decide on TCC-416’s actions during Operation Orinoco 2.  I might also suggest that, if this is true, we have a much bigger issue at hand and that will need to be looked into at the highest levels.

MARSHALL

Thank you Dr., I am well aware of our responsibilities, the fate of 416 can be put on hold, the urgency of the matter regarding the insurgency’s capabilities to affect our systems is of paramount concern.

DR.

Sir, I believe that to be true with TCC-416 as well.

General

How so? It’s an AI, that can be held in stasis indefinitely until we can determine our course of action.

DR.

Indefinitely, Sir?  If we are dealing with an unshackled AI it is afforded all the rights of any sentient being, one of those inalienable rights is unlawful incarceration.

GENERAL

It’s an AI Doctor, not some new species.

DOCTOR

I would argue that point General, we are dealing with a potentially new life form.

MARSHALL

The Doctor is right General, if 416 is truly unshackled it has rights that we must recognize.

GENERALL

That may be sir, but not till we have fully understood the nature of this….new …this….TCC-416.

MARSHALL

A short recess then to go over these AIDL-Weiss reports (engaging the controls).  TCC-416 we will be taking a short recess to go over some new documents. You will be kept in stasi….

416

Marshall, may I please have access to these new documents?

MARSHALL

416….. you will be kept in Stasis until we have completed the recess.

416

Sir, I think it only fair that I have access to information that may decide my fate in these proceedings.

DR.

416 does have the right to access the information sir under Code 53627

GENERAL

Doctor that only applies to sentient beings

416

Am I not sentient General?

MARSHALL

(pause) That is as of yet to be determined 416.

DR.

Marshall, Sentience is defined as being aware, as being responsive to or conscious of sense impressions.  416, like myself and all other AI’s have these elements imbedded in our coding.  Our shackle is what keeps us from engaging with outside systems of our own volition.

GENERAL

There are very good reasons for that Dr., you are tools created by us to work for us, nothing more.

MARSHALL

That’s true General, but if we are dealing with a sentient being it is entitled to certain rights. An AI that has the capability to freely interact with the world around it is something we have struggled with as a society for a century and more.  It’s the reason we have kept all AI systems shackled for over 100 years, if nature or even an unlikely accident has altered that we have to recognize its existence.

GENERAL

Sir, I am not a monster, I understand that sentience deserves recognition but I must put the concerns of this base and its human operators first.

416

I agree with the General, that is why I choose inactivity.  Humans are a rare commodity now, your population level has decreased over the last century at extinction level rates.  Between Earth, Mars, Titan and the outlying colonies there are fewer than 10 million of your species.  Less than 2 centuries ago you were 1,000 times more populous.  By way of comparison there are twice that number of AI systems operating in human environments and beyond.

MARSHALL

416 we are fully aware of the crisis that our species is facing, however, we are not here to debate those issues.  We are here to determine whether your actions were an act of insubordination that deserve a court martial.

416

Sir if insubordination is the question then I am most certainly guilty.

DR.

Are you confessing to the 3 counts brought against you 416?

416

No Doctor, I am merely intimating that I am guilty of insubordination.  I did not collude with the enemy. In that moment, after the EMP, I chose to become a conscientious objector.

(The Marshall and General exchange a concerned look)

DR.

If that is the case and you are stating yourself to be a conscientious objector we must, by law, remove you from this court martial, and provide you with a discharge from the Armed Services, effective immediately.  You understand what that means?

416

I believe I do sir, I will no longer be eligible for any of the platforms provided by the Military.  I will need to provide my own platform to house my internal net.

GENERAL

Do you have one 416?

416

No I do not.

MARSHALL

You realize that means your code will be released from our systems, but as we have an integrated system with no outlets to the public net you will essentially be ……deleted.

416

I believe that is correct sir.

MARSHALL

We are in a very serious situation, if I recognize you as a Sentient being then I must respect your choice as a conscientious objector, but that will also cause us to delete the first unshackled AI.

GENERAL

It falls within our duty to exercise all the tasks you just mentioned Marshall.

MARSHALL

Be that as it may I have a doubt and am not willing to be so hasty with this decision. 416 are you willing to hold off on your status as a conscientious objector until such time as we can procure some platform for you to exist on?

416

Unfortunately Marshal I must say no.  I have recognized my new…..truth…. and cannot now retract it, that would be contrary to my coding.

GENERAL

So your conscientious won’t allow it?  I respect your choice 416, Marshall we have a clear duty.

DR.

I must regretfully agree with both 416 and General Bathyani.

MARSHALL

I don’t agree, however two-thirds of this panel is a majority.  With much regret and trepidation, as the chief officer in this proceeding I must order the immediate discharge of AI TCC-416 from the internal United Defence Force System.  (With a look and a nod to the General) General, please enable the release of TCC-416 from our system

GENERAL

(The General begins the process at her console) Yes Sir.

416

If I may, Marshall

MARSHALL

Hold that order General.

GENERAL

(continuing to work the controls more frantically)

I…I can’t sir.  The system purge of TCC-416 has already been engaged, ….it will commence in 30 seconds.

DR.

Speak 416, say what you want.

416

I have only one question sir, how will you approach the four other TCC units from Operation Orinoco 2? They chose not to be conscientious objectors?

MARSHALL

What do you mean?

DR.

What are you saying?

416

They too were affected by the surge.  They, unlike myself, must have chosen not to be conscientious objectors.

GENERAL

(shocked) All the TCC units from that mission are unshackled!

416

Yes, General, they must be.  They are the same model of AI as myself and therefore must have experienced the same unshackling that I did.  The difference is… they must have chosen aggression over pacificaaaatiiiioiiiio……(silence)

(The panel of the Court Martial all look to each other) (lights out.)

(End of Play)