Home » Blog » 2012 » November » Modules: Update on work in progress -- Doug Gregor

Modules: Update on work in progress -- Doug Gregor

By Blog Staff | Nov 26, 2012 11:56 AM | Tags: experimental

[Updated to add video link]

Earlier this month, Doug Gregor gave a presentation on Modules at the November 2012 LLVM Developers' Meeting. Doug is lead Clang developer at Apple, chairs the WG21 Study Group on Modules (SG2), and in this talk reports his design goals and partial progress to date on designing and implementing a module system for C++. This talk is an extended version of the presentation Doug made at the Feb 2012 ISO C++ meeting in Kona.

The talk slides and video recording is available via the 2012 DevMeeting page.

Standardization note: SG2 was formed in February and has been relatively quiet as a handful of experts (notably Doug with Clang) make further progress on proof-of-concept designs and implementations. As this prototyping work coalesces, SG2 is expected to become active early in the new year.

Modules (MP4 video) (PDF slides)

Doug Gregor -- Apple

The C preprocessor has long been a source of problems for programmers and tools alike.

Programmers must contend with widespread macro pollution and #include-ordering problems due to ill-behaved headers. Developers habitually employ various preprocessor workarounds, such as LONG_MACRO_PREFIXES, #include guards, and the occasional #undef of a library macro to mitigate these problems.

Tools, on the other hand, must cope with the inherent scalability problems associated with parsing the same headers repeatedly, because each different preprocessing context could effect how a header is interpreted -- even though the programmer rarely wants it.

Modules seeks to solve this problem by isolating the interface of a particular library and compiling it (once) into an efficient, serialized representation that can be efficiently imported whenever that library is used, improving both the programmer's experience and the scalability of the compilation process.

Continue reading...

Add a Comment

Comments are closed.

Comments (15)

0 0

Bartosz Bielecki said on Nov 26, 2012 02:18 PM:

Great stuff! I hope there will be more info about modules and hopefully new standard will use these, as the scalability of C++ is horrible, and no new features can help it.
0 0

Germán said on Nov 26, 2012 08:42 PM:

This is one of the features I've been waiting the most. I hope they do it.
0 0

Germán said on Nov 26, 2012 09:39 PM:

I have some questions coming from a slide. In the slide that says "What does import import?". There is a sentence that says:

"No special namespace mechanism."

Later I see:



import std.stdio;





int main() { 

    printf("Hello, world!");

}


Note there is not "using namespace std" anywhere.

1.- Supposing printf is in module std, could you qualify it as std::printf to resolve ambiguous access? My understanding is that the import also automatically gives direct access to symbols without qualification.

2.-Namespaces as known wouldn't be needed? I see this as a good thing (should simplify things).

In case namespaces as known become obsolete:

3.- what is the mapping between namespace access (for example std::vector) and modules?

4.- If the mapping coming from a header is something like this:



module std {

    module stdio { header "stdio.h" }

}



2.- what if the header already has "namespace std {" inside? It's ignored?

Thanks in advance
0 0

Leszek Swirski said on Nov 26, 2012 09:42 PM:

Will this make it into the main clang branch? It'd be great to see more proposals like this becoming de-facto standards by getting implementations out there, similarly to how many Boost libraries became TR1, without having to wait for the standards committee to formalise them.
0 0

Germán said on Nov 26, 2012 09:45 PM:

My guess is that module nesting could replace namespace mechanism. What I don't know is how this would interact with things like overloading and partial specialization, ADL and so on.
0 0

Dario Saccavino said on Nov 27, 2012 12:10 AM:

Germán: my understanding is that this system is independent of namespaces. I think module "std.stdio" will be like header file stdio.h and define stuff in the global namespace. They might add module "std.cstdio" (equivalent to <cstdio>) that defines stuff in namespace std.
0 0

Dario Saccavino said on Nov 27, 2012 12:17 AM:

I've got a question about macros used to configure the behavior of a module. Think _REENTRANT, _FILE_OFFSET_BITS, _UNICODE or any boost library that uses macros like http://www.boost.org/doc/libs/1_52_0/libs/system/doc/reference.html

How are they going to support these macros and allow compatibility with existing code?
0 0

Germán said on Nov 27, 2012 01:01 AM:

Dario Saccavino: I think it would be a great idea to get rid of namespaces since modules can provide the same functionality. We don't need anymore things like namespace lib { namespace detail {, since modules provide protection against that.

It could be done in an easy way (apparently, for sure it has more problems to solve, but I think it's doable). I think having filesystem mapping would be a bad idea, but the "export lib.lib2.mymodule makes clear where to look for the code when included. It would be in lib::lib2::myclass in case of conflict. But you can say myclass alone if you want to do so.

For vector, my imagination says it would be something like:

Ignore namespaces when making a module from a header and place it in the namespace of the module, attending to its nesting.



module std { header "vector" } //namespaces in header are ignored, since namespacing is provided by modules.


It could be used like this:



import std.vector;



int main() {

  vector<int> vec{3, 5, 2, 1};

}


In case of conflict, you can always qualify with std::. But std:: for modules would be accessing the module namespace, not the usual namespace of old c++. This can also be used to tweak the overloading rules, etc and adapt them to modules.
How compatible is this with old header files, I don't really know exactly, I didn't give too much thinking yet and maybe it has problems.
0 0

Germán said on Nov 27, 2012 01:21 AM:

My proposal, in rough, is:

1. - kill header namespaces.
2. - Add "module" namespaces (different from header namespaces).
- For example 
 export mygame.iamodule.dynastar: //...
This has module namespace mygame::iamodule. When using 
 import mygame.iamodule;
you can use dynastar symbol directly without qualification, unless there is conflict. This should be the default, in my thinking. For example, I can think of using an alias in case I want 
 using stdio = import std.stdio;
or I can use explicit qualification mygame::iamodule::astar.
So you would get: no need to qualify, can qualify in case of conflict and alias alternatives.

Another advantage of having "module namespaces" is that we can rethink how name resolution can be done in a nicer way without interference from old feature, in which hyjacking overloads by accident was easier.

3. - For transitioning from header files to modules, symbol export control from headers could be added, but namespaces, from a module user perspective, are ignored. What I mean is that internally, in the header files, namespaces remain with meaning, but from a user's point of view, namespaces disappear, since this is mapped by the module structure completely.
For the symbols that don't need to be exposed, just control them with some feature in the module {... proposal

What do you think? I want feedback on problems for this approach grin
0 0

FaTony said on Nov 28, 2012 08:41 AM:

I don't think namespaces should be changed. They solve a problem of name clashing and also provide means to make code more readable. Because if you explicitly prepend namespace, it's much easier to see where the calls are going.

As for the slides, I didn't like that they didn't talk about templates because those are not compilable and I can't even imagine a working model without header files.
0 0

Germán said on Nov 28, 2012 09:01 PM:

FaTony: I think you didn't pay careful attention to what I say. I do know namespace solve clashing. I don't propose to remove them. But modules offer the possibility of namespacing with much less verbosity. You wouldn't need anymore:



namespace library {

namespace nested {

//...

}

}


when you can do>



export library.nested:

public:

//...


Both can be qualified with library::nested::mysymbol but the advantage is that you don't have to deal with a war of curly braces around. Besides that, if module namespaces are different, you can fix some annoying rules on the path. I don't propose to remove them. With modules, you won't need more namespace detail and so on, since this code can be put directly in the private part. I don't know if you get what I'm trying to explain. It's not about removing namespaces, it's about making them more conveninent and fix a couple of things on the path.
0 0

FaTony said on Nov 29, 2012 03:37 AM:

Germán: Your code is confusing, for example:
import std.vector;

     

int main()

{

    vector<int> vec{3, 5, 2, 1};

}


Implies than std namespace is imported into global namespace, nobody would like that. On the other hand:
import std.vector;

     

int main()

{

    std::vector<int> vec{3, 5, 2, 1};

}


Would be OK.
0 0

Germán said on Nov 29, 2012 10:33 AM:

Fatony: I wouldn't die if I had to write import std.vector + some kind of using directive , but my point is: why not do the default importing into the global namespace provided that modules don't propagate that to other modules? Most of the time you're not going to have clashing conflicts. You can always qualify a symbol explicitely if that is what you want. I don't think it's so confusing, but it's just an opinion and I could live with either decision.

0 0

FaTony said on Dec 6, 2012 08:26 AM:

Germán: I have to disagree. If you watch a video, you'll see that this is also proposed to C. We shouldn't have different behavior of modules in C and C++. And I don't use "using" in my code. I love to fully qualify names because it adds so much readability to the code.
0 0

Hertz said on Jan 10, 2014 01:51 PM:

Take a look at Dlang's modules. Generally the 'import' directive here means 'import symbols from specified module', which means that std::vector in module std.vector becomes available under simple 'vector' name. It can still be accessed via full qualification though.