Hi folks,

I was wondering if there was any indication as to when hotswap support might be available
for schema / shape changes to a class file. i.e. changes including new methods or fields.

Currently I understand that hotswapping classes only supports changes made to the implementation
of a code segment?

Thanks,
Mark.

 

 

The bug-id is 4910812 "Enhance Hot Code Replacement" if you want
to keep an eye on it.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4910812


That is correct.

At the Java language level:
http://java.sun.com/j2se/1.5.0/docs/guide/jpda/jdi/index.html

At the VM level:
http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html#RedefineClasses


For Additional Information:

"The JVMPI Transition to JVMTI",
http://java.sun.com/developer/technicalArticles/Programming/jvmpitransition/

"Creating a Debugging and Profiling Agent with JVMTI",
http://java.sun.com/developer/technicalArticles/Programming/jvmti/

"JVM TI Reference",
http://java.sun.com/j2se/5.0/docs/guide/jvmti/index.html

---

"Troubleshooting Guide for J2SE 5.0",
http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf

"The NetBeans Profiler Project", http://profiler.netbeans.org/

"Heap Analysis Tool", https://hat.dev.java.net/

"A Quick Start Guide To The Performance Tools",
http://developers.sun.com/tools/cc/articles/perftools_tip.html

---

 

 

Hello again.

I just updated the evaluation on 4910812, as follows:

A number of years ago we invested in research in this area.
The paper "Towards Flexible and Safe Technology for Runtime
Evolution of Java Language Applications" by Mikhail Dmitriev
provides an overview of this research:

http://www.experimentalstuff.com/Technologies/HotSwapTool/runtime-evol.pdf


At this time we have no plans to support arbitrary changes
to classes at runtime.

We are however conscious that a number of developers are
seeking the current restrictions to be relaxed in the
development environment. That is, for "fix & continue"
debugging developers are looking for ways to update classes
in the IDE/debugger without needing to restart the
application. In addition there are tools that do
instrumentation looking to do limited changes. We are not
adverse to making improvements but it is a topic that
requires further research. To that end it would be useful if
developers voting for this RFE could detail their specific
requirements. Are people really looking to do arbitrary
changes and instance conversion? We view type safety and
application consistency as very important. One approach is
to relax the restrictions and allow for some limited changes
- for example, the ability to add methods would address many
scenarios. We aren't making any commitments at this time but
we are looking for feedback.

This forum thread would be an appropriate place to add feedback:
http://forums.java.sun.com/thread.jspa?threadID=572396&tstart=0

We are interested in detailed information about specific features.

 

 

Here is another thread loosely related to this topic:
http://forums.java.net/jive/thread.jspa?forumID=23&threadID=481&messageID=12422#12422

 

 

Hi, what I've heard most is that people are looking for scripting language-like behaviour that allow for any change to be performed at run-time during development. Highest on my list is the ability to add/remove methods, add/remove member variables and change the inheritance and interface implementation signatures.

I'm not sure that instance conversion is needed. Most cases where I felt the need are web applications where many of the class instance are actually thrown away after each execution. They are mostly stateless. For the stateful parts (sessions, ...), instead of conversion, it might maybe be an option to add support for 'outdated instance listeners' that get notified when instances of classes have been replaced by a newer version. They could then take the required measures.

I think it's important to make a distinction between development-time comfort and run-time consistency and safety. Every developer will be aware that static instances and static repositories will needed special care and handling when their implementations are changes on-the-fly, however in 99,99% of the cases they just want to use the new instance of an updated and earlier defined class.

There is a discussion about this going on at TheServerSide also: http://www.theserverside.com/news/thread.tss?thread_id=37249#189142

 

 

I think in 80% of cases people just want the ability to add methods and instance fields on the fly while debugging, so I'm going to agree with timbell that having such abilities will adress many common scenarios . Having them implemented in Mustang would be a really good start :)

 

 

Hello,

It is about development, totally forget production. Production is fine as is now. I can imagine that this is very hard to implement all the way but that is not needed. Believe me that almost every developer I worked with is dying for this. Maybe it is possible to do it step by step so that every new release of Java adds a littlebit to it. My life and that of a whole lot of people would drammatically improve by just being able to add new methods and fields IN DEBUG MODE. I don't know what you mean with conversion but the updated class should also be reflected in existing instances. That cannot be hard and if it is I settle for less also.

Support for changes in inheritance would be nice but i rather have the supprt for new methods in the next release than having to wait longer and have it all. Please understand that this is so vital to the development proces (read time , read $) that the absence of this feature will eventually kill Java as a language, this is not a threat but a simple fact of rate*hour.

Thank you very much for giving me the opportunity to speak about this with somebody who can do something about it. I at least now have the feeeling that I am able to do something about my situation.

Kind regards,
Dennis Bekkering

 

 

Thanks all for your detailed feedback on the what and whys of extensions to hotswap support.

As a former Smalltalk developer I am well aware of the huge increase in development speed that you get when you can dynamically evolve the program. Some of these capabilities are currently available in existing Java platform debuggers which allow the bodies of mehods to change using the hotswap abilities of JPDA (JDI, JDWP & JVM TI). But these do not allow for true evolution, partly because of the limitations of hotswap.

There are two general mechanisms for class evolution:

Redefinition --

Foo f = new Foo();
f.bar(); // calls original bar
<redefinition happens here where body of bar is replaced>
f.bar(); // new bar is called

Update --

Foo f = new Foo();
f.bar(); // calls original bar
<update happens here where a new Foo is defined>
<as a result, the original Foo is renamed Old_Foo>
f.bar(); // f is an instance of Old_Foo, so original bar is called
f = new Foo();
f.bar(); // f now an instance of the updated Foo, so new bar is called

hotswaping as defined in the JVM TI spec and the interfaces that are built on it, do redefinition. This is what you want for instrumentation (such as profiling) and for most debugging situtations.

But to be able to change the fields of a class through redefinition requires all existing instances to be updated. Such an change is very difficult to implement, but worse, the semantics of the update are very dicey.

On the other hand, providing the ability to add, or even delete or change, methods via redefinition is non-trivial but probably doable.

Using the update machanism it is comparitively easy to update a class in a arbitrary way. There are some semantic issues. But the hard part here is the behavior of an application where some objects may be updated, and others are still running original versions of the class. No existing APIs are provided for class updating.

For some of you it sounds like updating would be what you are looking for. For others, I think you need/want extended redefinition.

Or maybe I've missed something.

Please clarify.

Thanks,
Robert Field
JVM TI Specification Lead
Serviceability Team
Sun Microsystems

 

 

Hi Robert,

thanks a lot for taking the time to look into this.

I'll just give you a typical use-case. The way we are writing large parts of our web applications is by driving everything from the beans that define our domain model. During development, it's very common to iteratively add properties, change their types, extract parts into common base classes, and so on. The framework we use adapts on-the-fly to these changes. However, in the current hotswap implementation, only the behaviour of the accessors and meta-data data methods can be changed. It would be a whole lot more comfortable to be able to modify the properties and class hierarchies on-the-fly too.

Something similar can be said for actions in web applications, once your main entry method grows too large, you want to split up the logic into separate methods or even refactor parts into common help classes. Each time you want to do that, you now need to restart the application while there's ho real purpose for doing this in terms or consistency. The actions are instantiated at each request and don't live any longer. You just want the latest version without the web application container that itself doesn't have any changed classes at all.

It's mainly about comfort and agileness during the iterative development process.

Thanks for listening,

Geert

 

 

Robert,
Right off the bat I want to say that simply supporting adding methods is not enough. The key strength with Java today is the tools, and all of the tools have advanced refactoring support. Adding, changing, and removing methods happens all the time and only allowing adding of methods would be almost as useless as HotSwap is today. Sorry to be so blunt, but this feature needs to be kept in context with the strengths of Java and how people work with Java.

I totally understand where you are coming from. From an engineering perspective, this feature clearly can be overwhelming to see, since there is no easy wasy to implement this short of a rich UI that guides the developer through changes that require decisions (ie: "field X was removed and field Y was added - was this a move or were they not related?").

I have to agree that this feature is absolutely needed for Java to keep pace. And, like another person said in the forums, this is all about development_mode, not production. As such, I think if you clearly document the behavior, you can make some assumptions that may or may not work for everyone, but will get us a lot further.

I also have to say that just adding support for redifining new instances does not help us. We can already do that via classloaders (http://wiki.opensymphony.com/display/WW/QuickStart), so I'd rather Sun didn't waste the energy on a feature we are already doing. We need to be able to redefine existing instances.

As you noted, fields pose the biggest problem in this area. I propose the following rule for dealing with them:

Default fields to null or their primitive field (if possible default them to their initialized value, but even that isn't absolutely needed, and I know can be more difficult)

As long as everyone understands this is a productivity feature and won't always work, then you'll be fine. At worst, the developer will need to restart, which is no different than today.

 

 

Hello Robert,

Extended redefinition by far for me. I know I think to simple about this but there is some kind of instance registration, what is so hard about iterating those instances and extend them?

And as plightbo mentioned it does not have to be perfect, if I lose a little time sometimes because of imperfectness I will not be half as upset as I am with the current situation. Actually if the JVM would give notice when things get ambiguous that would be perfect.;-) Anyway thanks for your quick response, I hope that is an indication of the speed with wich this will be implemented.

Kind regards,
Dennis

 

 

I realize that rule made no sense without a missing word or some example. Let's try again:

Default fields to null or their primitive field value

For example:

Person person ---> null
int ammount --> 0
* boolean bald --> false

Fields deleted would just simply be deleted - no fancy logic for trying to figure out where their data goes (such as a field rename).

 

 

I don't really understand what you mean with that. Do you mean that we should not be able to do

private boolean bald = true;

or that

private boolean bald;

will initially be false.

I think both should work, the latter is normal behaviour so I still don't really get your point. What other options do you see then?

 

 

Patrick, while in some way it's possible to achieve partly the same behavior with classloaders, it's still a big hack imho. It requires you to constantly check the modification times of the files, either at each usage, either in the background (as WebWork does). Doing these checks is extremely slow when you have a lot of files and not a fast HD. On laptops this just sucks away battery power. I thus think that hotswap would a huge improvement, even for this.

 

 

Robert,

If it would be done then where would that be? I have been browsing the mustang source. The C code is hard to get for me but in the java files I might have a chance of trying some stuff myself. Can you hand out any hints about where to do it? It is interesting anyway to browse the source of mustang.

Kind regards,
Dennis